CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

testland/llm-regression-suite-author

Use for any question or task about catching LLM quality regressions, including checking a new model version or provider against the old one and prompt tweaks that break other cases. Builds a versioned golden dataset, deterministic runs, exact, semantic, judge and rubric checks, and a pass-rate CI gate with a baseline diff across model versions.

Quality

90%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-crafted, highly actionable skill body: every step is executable, the workflow is clearly sequenced with explicit validation and feedback loops, and claims are tied to cited documentation. The remaining gaps are minor — some trimmable verbosity and bulk content (CI yaml, diff script) that could live in a scripts/ bundle the skill references but does not ship.

Suggestions

Move the regression_diff.py script into a bundled scripts/ file (the CI example already references it) and keep only the invocation in SKILL.md, fixing the mismatch where the path is used but no bundle exists.

Tighten the 'When to use' section and the opening paragraph, which partially restate the frontmatter description and the per-step context already conveyed by the step headings.

DimensionReasoningScore

Conciseness

The body is dense with tool-specific facts and executable code, with no explanation of concepts Claude already knows. Minor trimmable parts remain — the 'When to use' bullets partially restate the description, and the full 40-line CI yaml and 20-line diff script are inlined — so it is efficient but does not meet 'every token earns its place'.

4 / 5

Actionability

Every step provides copy-paste ready material: JSONL/CSV dataset formats, promptfoo provider YAML, DeepEval metric constructors, bash eval commands, the regression diff script, and a complete GitHub Actions workflow, covering both tool paths and the common cases. The only hand-waved step ('Generate actual_output from baseline model') is explicitly delegated to the per-tool skills.

5 / 5

Workflow Clarity

Six clearly sequenced steps with explicit validation checkpoints: baseline pass rate as the floor, dataset changes gated on a version bump plus CI baseline capture before becoming 'golden-current', a ratio threshold with non-zero exit on regression, and CI failing on threshold drop. Feedback loops exist ('--filter-failing' for targeted investigation) and the anti-patterns table acts as a checklist.

5 / 5

Progressive Disclosure

Good structure with well-signaled one-level-deep external references and a closing References section, but the skill is a single file with no bundle: the full CI workflow yaml and diff script are inlined bulk content, and the CI example invokes 'scripts/regression_diff.py' which is not actually bundled (no scripts/ directory exists). These minor organization gaps place it below the 'content appropriately split' anchor.

4 / 5

Total

18

/

20

Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description that explicitly states both trigger conditions and concrete capabilities with specific, natural phrasing. Its only weaknesses are slightly incomplete natural-term coverage and a broad 'any question or task' framing that creates minor overlap risk with generic LLM-evaluation skills.

DimensionReasoningScore

Specificity

Lists six-plus concrete actions — 'Builds a versioned golden dataset, deterministic runs, exact, semantic, judge and rubric checks, and a pass-rate CI gate with a baseline diff across model versions' — comprehensively covering the skill's pipeline. This matches the comprehensive-coverage anchor rather than the 'minor gaps' anchor at 4.

5 / 5

Completeness

Explicitly answers 'when' with 'Use for any question or task about catching LLM quality regressions, including checking a new model version or provider against the old one and prompt tweaks that break other cases' and 'what' with the concrete build actions. Both are present with concrete trigger phrases, matching the top anchor.

5 / 5

Trigger Term Quality

Includes natural user phrases such as 'catching LLM quality regressions', 'checking a new model version or provider against the old one', and 'prompt tweaks that break other cases'. Coverage is good but not comprehensive — common variations like 'model upgrade', 'evals', or 'before/after comparison' are missing, so it sits above anchor 3 but below anchor 5.

4 / 5

Distinctiveness Conflict Risk

The regression-gating niche (golden dataset, baseline diff, CI pass-rate gate) is distinct, but the broad framing 'any question or task about catching LLM quality regressions' leaves minor overlap risk with closely related evaluation skills like promptfoo-evaluation and deepeval-evaluation. Mostly distinct with minor overlap, matching anchor 4 rather than the minimal-conflict anchor 5.

4 / 5

Total

18

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

referenced_paths_exist

Referenced path issues: 2 missing

Warning

Total

15

/

16

Passed

Reviewed

On this page