CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

agent-regression-evals

Use before shipping any agent, prompt, model or tool change, or when a pass-rate drop needs a verdict. Runs repeated trials, gates candidate vs baseline with paired statistics in regression_gate.py, checks tool-call trajectories and finds flaky cases.

Quality

94%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

The canonical home for this skill is agent-regression-evals in alirezarezvani/claude-skills

SKILL.md
Quality
Evals
Security

Quality

Content

100%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is exemplary: an executable, verified command workflow with concrete data schemas, explicit validation and blocking semantics, sharp hard rules, and clean one-level-deep disclosure into real, substantive bundle files. Nothing in it teaches Claude what it already knows, and every section either instructs or constrains action.

DimensionReasoningScore

Conciseness

The body is ~75 dense lines with no padding: no concept tutorials, no re-explanation of statistics basics, and judgment stated in single sharp lines like "Dropping a case is the cheapest fake fix" and "A user meets every trial, so pass^k is the reliability number". The pass@k/pass^k paragraph defines the skill's own metric convention rather than teaching known material, so every token earns its place — the level-5 anchor; level 4 would require identifiable trimmable over-explanation, and none stands out.

5 / 5

Actionability

The workflow block gives three copy-paste-ready commands whose flags (--spec, --results, --k, --quarantine-out, --baseline, --candidate, --quarantine) verifiably match the actual argparse interfaces of the bundled scripts, plus a concrete JSONL row schema ("{\"case_id\": ..., \"trial\": ..., \"passed\": ..., \"trajectory\": ...}") and a per-script exit-code table. This is fully executable guidance covering the common cases — the level-5 anchor; there are no pseudocode or missing-detail gaps that would justify level 4.

5 / 5

Workflow Clarity

Steps 0–3 are clearly sequenced (run trials → grade trajectories → detect flakes → gate), validation is explicit via documented exit codes ("Exit 4 blocks the merge"), and the "gate blocks on" list is a five-condition checklist. Feedback loops for error recovery are present — "An expired entry re-enters the gate automatically", "the gate says how many cases it would take" for underpowered drops, and "It warns, and a human decides" — matching the level-5 anchor (explicit validation, feedback loops, checklists).

5 / 5

Progressive Disclosure

The body is a genuine overview: workflow, rules, a script table, and a references section where every path (3 references/, 3 scripts/, 4 assets/) verifiably exists, is one level deep, and is annotated with its content (e.g. "pass@k vs pass^k, per-case averaging, paired CI, McNemar, power (7 sources)"). Statistics detail is appropriately pushed to the reference files rather than inlined — the level-5 anchor; the level-4 anchor's "minor organization gaps" are absent.

5 / 5

Total

20

/

20

Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it answers what and when explicitly with concrete, script-level actions and natural trigger phrases. Its only weaknesses are missing common synonyms ("eval", "regression", "benchmark") and a broad opening trigger that creates minor overlap risk with adjacent governance/self-eval skills.

Suggestions

Add the domain's most natural user vocabulary to the description — e.g. "Use before shipping an agent, prompt, model, or tool change, or when eval pass rates drop and you need a verdict on whether it's a real regression" — so users who say "evals" or "regression" match without paraphrasing.

Sharpen the opening trigger to the evaluation context (e.g. "Use when evaluating whether a change to an agent, prompt, model, or tool regressed pass rate vs a baseline") to reduce overlap with prompt-governance and self-eval skills that also trigger on "shipping a prompt change".

Consider naming the paired-statistics verdict outcome in the trigger clause ("a pass-rate drop needs a verdict" already gestures at this) alongside the file-level vocabulary users would mention, such as "baseline.jsonl / candidate.jsonl".

DimensionReasoningScore

Specificity

The description lists four concrete actions — "Runs repeated trials", "gates candidate vs baseline with paired statistics in regression_gate.py", "checks tool-call trajectories", "finds flaky cases" — each naming its object and even the implementing script. This matches the level-5 anchor (multiple specific concrete actions, comprehensive for the domain); the level-4 anchor ("minor gaps in coverage") does not apply because trial-running, statistical gating, trajectory grading, and flake handling together cover the skill's full scope.

5 / 5

Completeness

Both questions are answered explicitly: the "what" is "Runs repeated trials, gates candidate vs baseline with paired statistics... checks tool-call trajectories and finds flaky cases", and the "when" is an explicit trigger clause — "Use before shipping any agent, prompt, model or tool change, or when a pass-rate drop needs a verdict". This mirrors the level-5 anchor example (concrete what plus "Use when..." with concrete trigger phrases); level 4 would require the when-clause to be only partly explicit, which it is not.

5 / 5

Trigger Term Quality

Natural phrases a user would actually say are present — "before shipping", "agent, prompt, model or tool change", "pass-rate drop", "flaky" — giving good keyword coverage. It falls short of the level-5 anchor (comprehensive synonyms and variations) because the most common user vocabulary for this domain — "eval(s)", "evaluation", "regression", "benchmark", "A/B" — never appears in the description text, so a user asking "did my evals regress?" must paraphrase to match.

4 / 5

Distinctiveness Conflict Risk

The capability framing (repeated trials, paired statistics, regression gating) carves out a clear niche, but the opening trigger "Use before shipping any agent, prompt, model or tool change" is broad and overlaps the invocation space of closely related skills (prompt governance, single-session self-eval). That is precisely the level-4 anchor ("mostly distinct; minor overlap risk with closely related skills"); level 5 would need the distinct trigger to carry the whole clause, as the PDF example's file-type triggers do.

4 / 5

Total

18

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository

alirezarezvani/claude-skills

Path

engineering/skills/agent-regression-evals/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.