CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/llm-regression-suite-author

Builds a versioned golden-dataset LLM regression suite for tracking quality across model upgrades: structures a versioned JSONL/CSV golden dataset, configures deterministic eval runs (temperature 0, seed), wires assertion layers (exact, semantic similarity, LLM-as-judge, rubric), enforces a pass-rate threshold with diff reporting vs the baseline model, and gates CI on regression. Use when upgrading an LLM provider model and needing a repeatable before/after quality gate, or when a prompt regression suite must track output quality across model versions over time.

76

Quality

95%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

92%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A strong, action-dense body: executable code at every step, a clearly sequenced workflow with validation and feedback loops, and disciplined avoidance of padding. The main weakness is progressive disclosure — it is a long monolithic file with no bundle/reference files, and the named regression_diff.py script is inlined rather than shipped under scripts/.

Suggestions

Move the regression_diff.py script into an actual scripts/regression_diff.py bundle file and reference it, instead of inlining the full source in the body, to support progressive disclosure and reuse.

Consider extracting the four assertion-layer examples (Step 3) and/or the GitHub Actions workflow (Step 6) into reference files (e.g. references/assertion-layers.md, references/ci-gating.yml) linked from a concise overview, reducing the SKILL.md footprint.

Add a short top-of-file navigation/index of the six steps so the monolithic body is easier to scan and selectively load.

DimensionReasoningScore

Conciseness

The body is code-forward and assumes Claude's competence — it does not patronizingly explain what JSONL, cosine similarity, or regression suites are, and uses inline doc-citation links to offload provenance rather than prose. It is not the score-2 case of unnecessary explanation; for a six-step complex skill each section earns its place.

3 / 3

Actionability

It provides fully executable, copy-paste-ready guidance throughout — promptfoo YAML configs, DeepEval GEval constructors, CLI commands, a runnable regression_diff.py script, and a GitHub Actions workflow — matching the score-3 anchor rather than the pseudocode/describes-but-doesn't-instruct lower anchors.

3 / 3

Workflow Clarity

Six steps are explicitly sequenced (structure dataset -> configure eval -> layer assertions -> capture baseline -> diff candidate vs baseline -> CI gating) with validation checkpoints: baseline-as-floor capture, a diff script that sys.exit(1) on regression, and a --filter-failing feedback loop for investigation, matching the score-3 anchor with explicit validation and error-recovery feedback.

3 / 3

Progressive Disclosure

The body is a large single-file document (~360 lines) with well-organized sections but no bundle files in references/, scripts/, or assets/; content that could be split — notably the inlined scripts/regression_diff.py referenced by name and the full assertion-layer examples — stays inline. This fits the score-2 anchor (some structure, content that should be separate is inline) rather than the score-3 anchor requiring well-signaled one-level-deep reference files.

2 / 3

Total

11

/

12

Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A high-quality description that pairs a dense, concrete capability list with an explicit Use-when trigger clause and a distinct, narrow niche. It reads in third person and avoids vague fluff or over-claims.

DimensionReasoningScore

Specificity

The description lists multiple concrete actions ('structures a versioned JSONL/CSV golden dataset, configures deterministic eval runs (temperature 0, seed), wires assertion layers (exact, semantic similarity, LLM-as-judge, rubric), enforces a pass-rate threshold with diff reporting... and gates CI on regression'), matching the score-3 anchor of multiple specific concrete actions rather than the domain-only score-2 anchor.

3 / 3

Completeness

It explicitly answers both what (the full build/gate workflow) and when via an explicit 'Use when upgrading an LLM provider model and needing a repeatable before/after quality gate, or when a prompt regression suite must track output quality across model versions over time' clause, matching the score-3 anchor rather than the score-2 anchor where 'when' is only implied.

3 / 3

Trigger Term Quality

Natural user-facing terms are covered ('upgrading an LLM provider model', 'before/after quality gate', 'prompt regression suite', 'output quality across model versions'), which a user would plausibly say when needing this skill; not merely technical jargon as in the score-1 anchor.

3 / 3

Distinctiveness Conflict Risk

The niche is narrow and specific — versioned golden-dataset regression across model upgrades with baseline diffing and CI gating — with triggers unlikely to fire for general eval skills; it is not generic like the score-1 anchor 'Helps with code and documents'.

3 / 3

Total

12

/

12

Passed

Validation

87%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation14 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

metadata_version

'metadata.version' is missing

Warning

referenced_paths_exist

Referenced path issues: 2 missing

Warning

Total

14

/

16

Passed

Reviewed

Table of Contents