CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

regression-consistency-checker

Checks whether a new version of a repository preserves the behavior observed by tests on the old version. Use this skill when comparing two versions of code to detect regressions, verify refactoring safety, validate bug fixes don't break existing functionality, or ensure backward compatibility. Detects differences in function outputs, exceptions, observable states, and performance between versions. Generates reports highlighting potential regressions (critical, high, medium, low severity), improvements, and areas requiring verification. Triggers when users ask to check for regressions between versions, compare test behavior across versions, verify behavior preservation, or validate that changes don't break existing tests.

70

Quality

88%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

73%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, largely executable regression-checking workflow with excellent sequencing and validation loops. Its weaknesses are padding (duplicate script usage, generic best-practices and comparison-strategy sections, an inline report template) and one pseudocode call to a nonexistent function, which together hold conciseness and actionability below the top anchors.

Suggestions

Delete the "Helper Script" section — it repeats the exact three `compare_results.py` commands already shown in step 4 — and trim "Comparison Strategies" snippets like `old_output == new_output` that Claude obviously knows.

Cut or move content duplicated with the bundle: the "Regression Types" summary restates detection_strategies.md, and the full sample report (lines 143-196) could become a short template or a references/report_template.md file.

Replace the pseudocode `compare_ignoring_fields(old, new, ['timestamp', 'id'])` with a real invocation or drop it, and note the `pytest-json-report` plugin prerequisite next to the first pytest command.

DimensionReasoningScore

Conciseness

The core workflow is command-driven and efficient, but several sections pad the document: "Best Practices" recites generic advice Claude already knows, "Comparison Strategies" includes trivial snippets like `old_output == new_output`, and the "Helper Script" section repeats the exact three `compare_results.py` invocations already shown in step 4. Not verbose enough overall for a 2 (mostly efficient core workflow), but more trimming needed than a 4 allows.

3 / 5

Actionability

Mostly executable, copy-paste-ready commands (`pytest --json-report --json-report-file=old_results.json`, `git bisect run pytest path/to/failing_test.py`, `python scripts/compare_results.py old_results.json new_results.json --tolerance 0.001`) backed by a real bundled script, with both pytest and Jest covered. Minor gaps keep it below 5: `compare_ignoring_fields(old, new, ['timestamp', 'id'])` is pseudocode for a nonexistent function, and the `pytest-json-report` plugin dependency is never noted.

4 / 5

Workflow Clarity

A clear 8-step sequence (prepare versions → baseline → new run → compare → analyze → investigate → document → fix/accept) with explicit validation ("Verify baseline stability", "Run multiple times to check stability", compare via script) and feedback loops for error recovery (git bisect for root cause; fix → re-run tests → re-compare in step 8). Every risky step has a corresponding checkpoint, matching the top anchor.

5 / 5

Progressive Disclosure

Good structure with both bundle files real, one level deep, and clearly linked ("Review strategies in [detection_strategies.md](references/detection_strategies.md)"; `scripts/compare_results.py` used throughout). Below 5 because the body's "Regression Types" Quick Reference duplicates content that already lives in detection_strategies.md, and the ~54-line sample report template is inline content that would fit better in a reference file.

4 / 5

Total

16

/

20

Passed

Description

96%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description that explicitly and concretely states both what the skill does (detects output, exception, state, and performance differences; generates severity-graded reports) and when to use it, with comprehensive natural trigger phrases. The only weakness is mild overlap risk from broad validation phrasing, and the duplicated trigger clause ("Use this skill when..." plus "Triggers when...") adds slight redundancy.

DimensionReasoningScore

Specificity

"Checks whether a new version of a repository preserves the behavior observed by tests on the old version", "Detects differences in function outputs, exceptions, observable states, and performance between versions", "Generates reports highlighting potential regressions (critical, high, medium, low severity), improvements" — multiple specific concrete actions (four detection categories plus severity-graded reporting) with comprehensive coverage of the skill's behavior, matching the top anchor rather than the 'several actions, minor gaps' anchor below it.

5 / 5

Completeness

Explicitly answers both questions with concrete trigger phrases: the 'what' ("Detects differences in function outputs, exceptions, observable states, and performance... Generates reports...") and the 'when' twice over ("Use this skill when comparing two versions of code to detect regressions, verify refactoring safety..." and "Triggers when users ask to check for regressions between versions..."). This is the anchor-5 pattern, well above anchor 4's 'when could be more explicit'.

5 / 5

Trigger Term Quality

"Check for regressions between versions, compare test behavior across versions, verify behavior preservation, verify refactoring safety, ensure backward compatibility, validate that changes don't break existing tests" — comprehensive natural-phrase coverage with the synonyms and phrasings a user would actually say (regressions, refactoring, backward compatibility, behavior preservation).

5 / 5

Distinctiveness Conflict Risk

The cross-version regression niche is clear with distinct triggers ("regressions between versions", "behavior preservation"), but broad phrases like "validate that changes don't break existing functionality" carry minor overlap risk with general test-running or code-review skills. Mostly distinct with minor overlap risk (anchor 4) rather than the minimal-conflict anchor 5.

4 / 5

Total

19

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

ArabelaTso/Skills-4-SE

Path

skills/regression-consistency-checker/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.