CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

phoenix-evals

Build, run, and validate evaluators for AI/LLM applications with Phoenix, in Python or TypeScript (including Vercel AI SDK apps) — code-based checks and LLM-as-a-judge, run over exported traces or dataframes or as Phoenix experiments, gate them in CI, and validate judges against human-labeled data.

Quality

83%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

82%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

An exemplary index-style SKILL.md: lean, well-structured, and fully verifiable — every link resolves to a real file one level deep. Weaknesses are minor: one orphaned bundle file, brace-pattern pseudo-links in the Workflows section, and validation steps framed as sequence positions rather than explicit pass/fail gates with recovery loops.

Suggestions

Link references/experiments-overview.md from the Quick Reference table (e.g., under 'Create dataset' or 'Run experiment') so it is discoverable.

Replace brace patterns like "evaluators-{code|llm}-{python|typescript}" in Workflows with explicit links to the relevant reference files, matching the Quick Reference style.

Make the judge-validation checkpoint an explicit gate in the Building Evaluator workflow (e.g., 'validation-evaluators-python → if accuracy <80%, rework the template/prompt and re-validate') to add a recovery loop.

DimensionReasoningScore

Conciseness

The body is a lean index: terse tables ("Error analysis first | Can't automate what you haven't observed", "Validate judges | >80% accuracy, >70% TPR/TNR") with zero padding and no explanation of concepts Claude already knows. Every section (Quick Reference, Workflows, Reference Categories, Key Principles) earns its tokens; this matches the 'lean and efficient; every token earns its place' anchor.

5 / 5

Actionability

Concrete, specific guidance throughout — a task→file map with exact paths, five sequenced workflow itineraries, and a hard numeric acceptance criterion (">80% accuracy, >70% TPR/TNR") for judge validation. It is not a 5 because nothing in the body is copy-paste executable (all code lives one hop away) and some workflow steps use brace patterns like "evaluators-{code|llm}-{python|typescript}" instead of explicit links; it is above the 'incomplete guidance' anchor because every task resolves to an exact, verified-to-exist reference file.

4 / 5

Workflow Clarity

Five named workflows (Starting Fresh, Building Evaluator, RAG Systems, Gating CI, Production) present clear file-to-file sequences, and validation appears as an explicit terminal step in the Building Evaluator and Gating CI flows with concrete thresholds. It is not a 5 because checkpoints are positional rather than explicit gates — no 'if validation fails, do X' recovery loops — and the Starting Fresh / RAG workflows end without any validation step.

4 / 5

Progressive Disclosure

The body is exactly an overview routing to 36 one-level-deep reference files, all of which exist, with a task table and prefix taxonomy for discovery; sibling cross-links (e.g., validation.md → validation-evaluators-python.md) stay within one flat directory. It is not a 5 because references/experiments-overview.md exists in the bundle but is unreachable from SKILL.md or any other file — a real navigation gap — and the Workflows section's brace-pattern steps are slightly less clearly signaled than the fully linked Quick Reference table.

4 / 5

Total

17

/

20

Passed

Description

75%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A highly specific, distinctive description that thoroughly states what the skill does across the evaluator lifecycle. Its main weakness is the absence of any 'Use when...' trigger clause, which caps completeness and leaves activation to inference. Third-person imperative voice is used correctly with no person penalties.

Suggestions

Append an explicit trigger clause such as: 'Use when building or running evals for LLM apps, setting up LLM-as-a-judge, exporting and evaluating Phoenix traces, or gating AI quality in CI.'

Add natural synonyms users actually say — 'evals', 'evaluations', 'AI/LLM testing', 'RAG evals', 'hallucination/faithfulness checks' — to broaden trigger coverage.

Optionally name concrete trigger artifacts users mention (e.g., 'a DataFrame of LLM outputs', 'exported OpenTelemetry spans') to sharpen when-conditions further.

DimensionReasoningScore

Specificity

The description enumerates many concrete actions — "Build, run, and validate evaluators", "code-based checks and LLM-as-a-judge, run over exported traces or dataframes or as Phoenix experiments", "gate them in CI", and "validate judges against human-labeled data" — comprehensively covering the skill's scope. This matches the anchor for multiple specific concrete actions with comprehensive coverage; it is above the 'several specific actions, minor gaps' anchor because no significant capability area of the skill is left unmentioned.

5 / 5

Completeness

The 'what' is explicit and detailed, but there is no 'Use when...' clause or equivalent explicit trigger guidance anywhere in the description; usage occasions are only weakly implied by the capability list. Per the rubric guideline, a missing 'Use when' clause caps completeness at 3, which also matches the 'clear what, when missing' anchor; it is not a 4 because no explicit 'when' statement exists at all.

3 / 5

Trigger Term Quality

Good keyword coverage with natural terms users would say: "evaluators", "LLM-as-a-judge", "traces", "CI", "Python or TypeScript", "Vercel AI SDK". It is below the comprehensive-synonyms anchor because common variations users actually say are missing — notably "evals" (the near-universal shorthand), "evaluations", "testing AI apps", and domain terms like "RAG" or "faithfulness" that dominate the skill's content.

4 / 5

Distinctiveness Conflict Risk

It names a specific niche tool ("with Phoenix", "Phoenix experiments") plus specific stacks ("Python or TypeScript (including Vercel AI SDK apps)"), giving it a clear niche with distinct triggers and minimal overlap risk with generic evaluation or testing skills. It is above the 'minor overlap risk with closely related skills' anchor because the tool name and language qualifiers make mis-triggering unlikely.

5 / 5

Total

17

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

Arize-ai/phoenix

Path

.agents/skills/phoenix-evals/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.