CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

phoenix-evals-new-metric

Add a new built-in (pre-built) LLM classification evaluator to the Phoenix evals libraries, and optionally promote it to a server-side dataset evaluator. Use when working in the Phoenix repo and asked to add a new built-in eval, pre-built metric, builtin evaluator, or LLM-as-a-judge template that ships with arize-phoenix-evals / @arizeai/phoenix-evals, or to promote an existing one to the dataset evaluators UI. Do not use for writing a user's own custom evaluator for their app (use phoenix-evals).

75

Quality

94%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A strong, highly actionable workflow guide: every step names exact files, commands, and canonical examples, and validation checkpoints plus an iterate-to-target benchmark loop are built in. The only weaknesses are minor — some tightening possible in the promotion section, and a couple of sections could be split into reference files.

Suggestions

Tighten the 'Promoted dataset evaluators' section: the mechanism explanation, the grep command, and the current examples list could be condensed since the 'Key Decision Points' and Step 0 already point back to it.

Consider moving the promotion criteria, current promoted/library-only example lists, and the Step 8 docs table into a reference file (e.g. references/promotion.md) to keep SKILL.md a lean overview with one-level-deep references.

DimensionReasoningScore

Conciseness

The body is dense with repo-specific facts (exact paths, commands, canonical example files) and explains nothing Claude already knows, but the ~25-line "Promoted dataset evaluators" section and the "labels"/"substitutions" decision points restate promotion logic that could be tightened. Fits anchor 4 ("efficient; minor instances of over-explanation that could be trimmed"), not 5.

4 / 5

Actionability

Fully executable guidance throughout: exact file paths for every artifact, runnable commands (make codegen-prompts, cd js && pnpm build, the grep one-liner, the vitest invocation), and named canonical files to copy from (correctness.py, createCorrectnessEvaluator.ts, faithfulness.mdx). Copy-paste ready and covers the common cases, matching the 5 anchor.

5 / 5

Workflow Clarity

A linear Steps 0–8 pipeline with explicit checkpoints ("Verify the generated files look correct before moving on", "Fix any TypeScript errors before proceeding"), a genuine feedback loop (failed-examples printer → adjust prompt or benchmark → iterate to >80% accuracy), a completion checklist, and a retrospection step for drift. Matches the 5 anchor including error-recovery loops.

5 / 5

Progressive Disclosure

No bundle files exist and the ~210-line body is well-sectioned with the sequential workflow content appropriately inline; however, the promotion criteria/examples and the docs-section table are separable reference material that could live in a reference file, and there are no external references at all. Fits anchor 4 (good structure, minor organization gaps), not 5 which requires well-signaled one-level-deep references or a short skill needing none.

4 / 5

Total

18

/

20

Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

An exemplary description: concrete third-person actions, explicit 'Use when' triggers with enumerated synonyms, and an explicit negative scope that separates it from the adjacent custom-evaluator skill. Every judged dimension lands on the top anchor without over-claiming.

DimensionReasoningScore

Specificity

"Add a new built-in (pre-built) LLM classification evaluator to the Phoenix evals libraries" and "promote it to a server-side dataset evaluator" name multiple concrete actions in third person that comprehensively cover everything the skill does. Not 4: there is no gap in coverage of the skill's scope.

5 / 5

Completeness

Both questions are answered explicitly: the "what" (add a built-in evaluator, optionally promote it) and the "when" ("Use when working in the Phoenix repo and asked to add a new built-in eval... or to promote an existing one to the dataset evaluators UI"), plus a concrete exclusion clause. Matches the 5 anchor example; not 4 because the triggers are fully explicit.

5 / 5

Trigger Term Quality

"built-in eval, pre-built metric, builtin evaluator, or LLM-as-a-judge template" explicitly enumerates the synonyms a user would naturally say, alongside package names (arize-phoenix-evals / @arizeai/phoenix-evals) and "dataset evaluators UI". Not 4: common variations are already listed rather than a few missing.

5 / 5

Distinctiveness Conflict Risk

Scoped to the Phoenix repo and named evals packages with a clear niche, and "Do not use for writing a user's own custom evaluator for their app (use phoenix-evals)" actively disambiguates from the nearest competing skill. Minimal conflict risk; not 4 because the overlap risk is explicitly foreclosed.

5 / 5

Total

20

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

metadata_field

'metadata' should map string keys to string values

Warning

Total

15

/

16

Passed

Repository

Arize-ai/phoenix

Path

.agents/skills/phoenix-evals-new-metric/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.