phoenix-evals-new-metric

Create a new built-in classification evaluator for Phoenix evals. Use this skill whenever the user asks to create a new eval, build a new metric, add a new builtin evaluator, create an LLM-as-a-judge metric, or add a new classification evaluator to Phoenix.

Quality

85%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

81%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, actionable linear workflow with strong validation checkpoints, feedback loops, and a closing checklist plus retrospection. The main improvements would be trimming minor explanatory asides and optionally splitting the docs section structure into a reference file.

Suggestions

Trim contextual asides (e.g., 'This affects how Phoenix displays the metric in the UI', 'the placeholders are the contract between the evaluator and the caller') to tighten conciseness.

Consider inlining a minimal YAML config skeleton alongside the 'read an existing config' pointer so the Step 1 action is copy-paste ready without first opening another file.

Optionally move the 12-section documentation structure list into a reference file (e.g., references/docs-template.md) and link to it, improving progressive disclosure for the longest inlined section.

Dimension	Reasoning	Score
Conciseness	The body is largely efficient and assumes Claude's competence (no basic-concept padding), but contains a few explanatory asides like 'This affects how Phoenix displays the metric in the UI' and 'the placeholders are the contract between the evaluator and the caller' that could be trimmed.	4 / 5
Actionability	It provides concrete copy-paste commands (make codegen-prompts, cd js && pnpm build, phoenix serve, pnpm --filter ...) and exact file paths for every artifact, but the actual code-authoring steps point to canonical example files to read rather than inlining templates, leaving minor gaps.	4 / 5
Workflow Clarity	A clearly linear Step 0–8 sequence with explicit validation checkpoints ('Verify the generated files look correct before moving on', 'Fix any TypeScript errors before proceeding'), an iterate-until->80%-accuracy feedback loop, a final checklist, and a retrospection step — matching the anchor for explicit validation with error-recovery feedback loops.	5 / 5
Progressive Disclosure	Well-organized into clearly headed steps with one-level-deep, well-signaled pointers to repo template files ('Read correctness.py in that directory — it's the canonical example') and no nested reference chains, though no bundle files exist and some longer sections (e.g., the 12-point docs structure) are inlined rather than split out.	4 / 5
	Total	17 / 20 Passed

Description

90%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, well-targeted description that clearly states what the skill does and when to use it with comprehensive natural trigger phrases. The only weakness is that its listed actions are synonymous variations of a single 'create' operation rather than distinct capabilities.

Dimension	Reasoning	Score
Specificity	It names the domain ('built-in classification evaluator for Phoenix evals') and one concrete action ('Create'), but the rest of the list ('build a new metric', 'add a new builtin evaluator', 'create an LLM-as-a-judge metric', 'add a new classification evaluator') are synonyms for the same create action rather than distinct capabilities, so coverage is not comprehensive.	3 / 5
Completeness	It explicitly answers both 'what' ('Create a new built-in classification evaluator for Phoenix evals') and 'when' ('Use this skill whenever the user asks to create a new eval, build a new metric...') with concrete trigger phrases.	5 / 5
Trigger Term Quality	It covers natural trigger phrases a user would actually say — 'create a new eval', 'build a new metric', 'add a new builtin evaluator', 'LLM-as-a-judge metric', 'classification evaluator' — with comprehensive synonyms across the trigger space.	5 / 5
Distinctiveness Conflict Risk	The Phoenix-specific 'built-in classification evaluator' niche and its targeted create-evaluator triggers make it clearly distinguishable from other skills with minimal conflict risk.	5 / 5
	Total	18 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
metadata_field	'metadata' should map string keys to string values	Warning

	Total	15 / 16 Passed

Repository: Arize-ai/phoenix
Path: .agents/skills/phoenix-evals-new-metric/SKILL.md
Commit: e127482

Reviewed: 6 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.