CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

phoenix-evals

Build, run, and validate evaluators for AI/LLM applications with Phoenix, in Python or TypeScript (including Vercel AI SDK apps) — code-based checks and LLM-as-a-judge, run over exported traces or dataframes or as Phoenix experiments, gate them in CI, and validate judges against human-labeled data.

66

Quality

83%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

82%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

This is a well-structured navigation-hub SKILL.md: token-efficient, task-oriented, with all detail correctly pushed to one-level-deep reference files and useful project-specific principles. The main gaps are one orphaned bundle file (experiments-overview.md), wildcard path patterns instead of literal filenames, and only implicit validate-retry loops.

Suggestions

Link experiments-overview.md from the Quick Reference table (e.g., under a 'Run experiment' or 'Experiments overview' row) so every bundle file is discoverable from SKILL.md.

Replace wildcard patterns like 'evaluators-{code|llm}-{python|typescript}' in the Workflows section with the literal filenames (or at least spell out the four concrete variants once) so each workflow step is unambiguous.

Add one explicit feedback-loop line to the workflows that gate CI (e.g., 'validation fails → revise evaluator/template → re-validate against labels') to make the retry semantics concrete.

DimensionReasoningScore

Conciseness

The 70-line body is a lean routing hub: a task→file table, five short workflow chains, a category table, and terse principles ('Code first, LLM for nuance, validate against humans'). There is no padding and no explanation of concepts Claude already knows — every line directs navigation or states a project-specific principle.

5 / 5

Actionability

Concrete task→file routing, named workflow sequences, and quantitative thresholds ('>80% accuracy, >70% TPR/TNR') give mostly executable guidance, and absence of code is acceptable for an instruction-hub skill per the rubric's scoring notes. It falls short of 5 because wildcard patterns like 'evaluators-{code|llm}-{python|typescript}' and 'integrations-{pytest|vitest-jest}' require Claude to interpolate rather than follow literal paths.

4 / 5

Workflow Clarity

Five named workflows (Starting Fresh, Building Evaluator, RAG Systems, Gating CI, Production) present clear ordered sequences, with validation as an explicit terminal step in the Building Evaluator chain and Key Principles supplying checkpoint criteria and gating semantics ('assert/expect hard invariants (CI red); log LLM-judge quality signals and gate the aggregate'). Not 5: feedback loops (validate → fix → retry) are only implicit, and wildcard file placeholders leave some steps less than fully defined — matching the anchor 'clear sequence with most checkpoints present; minor validation gaps' rather than 'explicit validation steps with feedback loops'.

4 / 5

Progressive Disclosure

Scored against the actual bundle: 38 of 39 reference files are reachable via clearly signaled one-level-deep markdown links from a task-oriented Quick Reference table plus a category table — good hub-and-spoke structure. Not 5: 'experiments-overview.md' exists in references/ but is not linked anywhere in SKILL.md (a discovery gap), and a few references cross-link siblings, so the anchor 'minor organization gaps' at 4 fits better than 'easy navigation' at 5.

4 / 5

Total

17

/

20

Passed

Description

75%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, specific, third-person description with comprehensive capability coverage and a clear Phoenix niche. Its one structural weakness is the missing 'Use when...' trigger clause, which caps completeness, and a handful of natural synonyms ('evals', 'benchmarks') are absent.

Suggestions

Append an explicit trigger clause, e.g.: 'Use when the user mentions evals, LLM-as-a-judge, Phoenix experiments, or wants to add evaluation gates to CI.'

Add the common shorthand 'evals' (and 'benchmarks' if accurate) as trigger terms, since practitioners rarely say 'evaluators' in full.

Consider naming 'datasets' explicitly as a trigger for the experiment/dataset-creation workflows.

DimensionReasoningScore

Specificity

The description lists multiple specific concrete actions with comprehensive coverage: 'Build, run, and validate evaluators', 'code-based checks and LLM-as-a-judge, run over exported traces or dataframes or as Phoenix experiments', 'gate them in CI', and 'validate judges against human-labeled data' — spanning build, run, gate, and validate in both supported languages. It is written in third-person imperative form ('Build, run, and validate'), so no voice penalty applies.

5 / 5

Completeness

The 'what' is clearly and comprehensively stated, but there is no 'Use when...' clause or any equivalent explicit trigger guidance ('Use it when...'), which per the judging guidelines caps completeness at 3. It is not 4 because the 'when' is entirely absent rather than present-but-imprecise, and not 2 because the 'what' is concrete and detailed.

3 / 5

Trigger Term Quality

Good keyword coverage includes natural terms users would say: 'evaluators', 'LLM-as-a-judge', 'traces', 'CI', 'Phoenix', 'Python', 'TypeScript', and 'Vercel AI SDK'. It misses a few common variations and synonyms — users frequently say 'evals', 'experiments' (only 'Phoenix experiments' appears), 'benchmarks', or 'datasets' — so it matches anchor 4 ('good keyword coverage; a few natural terms missing') rather than the comprehensive synonym coverage of anchor 5.

4 / 5

Distinctiveness Conflict Risk

The phrase 'with Phoenix' plus distinctive anchors like 'LLM-as-a-judge', 'Phoenix experiments', and 'validate judges against human-labeled data' establish a clear niche with minimal conflict risk against generic testing or observability skills.

5 / 5

Total

17

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

Arize-ai/phoenix

Path

.agents/skills/phoenix-evals/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.