CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

evaluators

Author or refine a Phoenix evaluator — code or LLM-as-a-judge — that scores a run's output. Trigger when the user wants to create a new evaluator, improve an existing one's logic or rubric, choose labels, or decide what to measure on a dataset or experiment. Do NOT trigger on: (1) manual prompt drafting (use `playground`), (2) running or comparing experiments themselves (use `experiments`), (3) cross-trace failure diagnosis with no evaluator in scope (use `phoenix-error-analysis`).

70

Quality

88%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

81%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A disciplined, well-sequenced authoring workflow with genuine validation and feedback loops, written for an intelligent reader without padding. The main gaps are minor: some redundancy between the loop and the avoid-list, and no worked example (code evaluator or judge prompt) to anchor the concrete steps.

Suggestions

Add one minimal worked example — a short code-evaluator signature or a compact judge-prompt rubric naming labels and evidence — to lift actionability to copy-paste-ready for the most common case.

Trim the duplicated 'persistence is a separate save' guidance (loop step 7 vs. Things To Avoid) and let the avoid-list cover only points not already in the loop.

Consider moving the judgment-structure ladder or field-topology details into a one-level-deep reference file (e.g., references/judgment-structures.md) to keep SKILL.md a lean overview with clearly signaled references.

DimensionReasoningScore

Conciseness

The body is dense and assumes Claude's competence (no boilerplate explanations of Phoenix, evaluators, or basic concepts), but 'persistence is a separate save' is stated nearly verbatim in both loop step 7 and Things To Avoid, and the avoid-list otherwise restates loop points — minor redundancy that could be trimmed. It is not a 5 because not every token earns its place, and not a 3 because there is no padded or unnecessary explanation.

4 / 5

Actionability

Guidance is concrete for an instruction-only skill: it names the exact run fields to inspect ('a chat-style messages array, assistant content parts, tool_calls, or a function_call'), specific deterministic primitives ('exact-match, contains, regex-match, json-distance...'), and a concrete binary label design. It falls short of 5 because there is no worked example — no sample code-evaluator signature, input declaration, or judge-prompt rubric — leaving minor gaps for the most common authoring cases.

4 / 5

Workflow Clarity

The 7-step Authoring Loop has an explicit validation checkpoint ('Calibrate against several representative cases covering the named failure modes'), a feedback loop for error recovery ('When a case mislabels, change one thing — rubric, logic, labels, or case'), and a clear exit condition ('Iterate until the representative cases label correctly'), matching the top anchor. This is a design workflow, not a destructive or batch operation, so no cap applies.

5 / 5

Progressive Disclosure

Sections are well-organized and content is appropriately placed inline for a process skill with no bundle files, but at ~85 lines it exceeds the simple-skill threshold and makes no use of one-level-deep reference files for detail that could be split out (e.g., the judgment-structure ladder or field-topology specifics). Good structure with minor organization gaps rather than the clear reference-split of the top anchor.

4 / 5

Total

17

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: concrete actions, explicit trigger and anti-trigger guidance, and clear boundaries against sibling skills. The only weakness is slight thinness in natural-term synonym coverage for how users might phrase evaluator/judge requests.

DimensionReasoningScore

Specificity

The description lists multiple concrete actions — 'Author or refine', 'improve an existing one's logic or rubric', 'choose labels', 'decide what to measure on a dataset or experiment' — with comprehensive coverage of the skill's scope and no noticeable gaps, matching the comprehensive-coverage anchor.

5 / 5

Completeness

It explicitly answers both 'what' ('Author or refine a Phoenix evaluator — code or LLM-as-a-judge — that scores a run's output') and 'when' ('Trigger when the user wants to create a new evaluator...') with concrete trigger phrases plus explicit non-triggers, matching the top anchor exactly.

5 / 5

Trigger Term Quality

Natural phrases users would say are well covered ('create a new evaluator', 'improve an existing one's logic or rubric', 'choose labels', 'decide what to measure'), but a few natural variations users might use (e.g., 'judge', 'grading criteria', 'scoring metrics') are missing, placing it just below the comprehensive-with-synonyms anchor.

4 / 5

Distinctiveness Conflict Risk

It carves out a clear niche and explicitly disambiguates against sibling skills ('use playground', 'use experiments', 'use phoenix-error-analysis'), giving minimal conflict risk.

5 / 5

Total

19

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository

Arize-ai/phoenix

Path

src/phoenix/server/agents/prompts/skills/evaluators/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.