CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/model-performance-regression-gate

Computes held-out metrics (accuracy, F1, AUC, RMSE) for a retrained model and compares them against the current production model, failing promotion when any metric regresses beyond a configured tolerance. Adds per-segment checks via Deepchecks WeakSegmentsPerformance so a model that improves globally but regresses on a key slice is still blocked. Use when a retrained model is a candidate for promotion and the CI pipeline must enforce a per-metric pass/fail gate before the artifact is pushed to the model registry.

79

Quality

99%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

100%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured CI gate skill: executable code at every step, explicit pass/fail validation checkpoints, and a single well-signaled reference for tunables. Content is operational rather than tutorial-style and appropriately split between the runnable gate and its configuration reference.

DimensionReasoningScore

Conciseness

Lean operational prose that assumes Claude's competence — no basic-concept padding about what models or metrics are — and every section (tolerance direction, segment behavior, limitations) earns its place for a genuinely multi-step gate.

3 / 3

Actionability

Each step ships fully executable, copy-paste-ready Python with concrete imports and a complete GitHub Actions YAML; the regression-vs-classification variant is shown inline rather than left as pseudocode.

3 / 3

Workflow Clarity

Steps 1–8 are clearly sequenced with explicit validation checkpoints (per-metric FAIL, segment gate FAILED, train-test degradation gate FAILED), each printing a result and calling sys.exit(1); the anti-patterns and limitations tables provide error-recovery guidance.

3 / 3

Progressive Disclosure

SKILL.md is a runnable overview; the only externalized detail — YAML threshold config and full metric signatures — lives in references/gate-config-and-metrics.md, a real one-level-deep file signaled with explicit links in Steps 3 and 8.

3 / 3

Total

12

/

12

Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A precise, third-person description that names concrete actions, gives an explicit 'Use when' trigger, and carves out a clear niche from neighboring evaluation/monitoring skills. It satisfies every dimension at the top anchor with no verbosity or over-claiming.

DimensionReasoningScore

Specificity

Lists multiple concrete actions ('Computes held-out metrics (accuracy, F1, AUC, RMSE)', 'compares them against the current production model', 'failing promotion when any metric regresses beyond a configured tolerance', 'Adds per-segment checks via Deepchecks WeakSegmentsPerformance').

3 / 3

Completeness

Clearly answers both 'what' (compute and compare held-out metrics, per-segment checks, block promotion) and 'when' via an explicit 'Use when a retrained model is a candidate for promotion and the CI pipeline must enforce a per-metric pass/fail gate' clause.

3 / 3

Trigger Term Quality

Uses natural terms an ML engineer would say ('retrained model', 'production model', 'promotion', 'CI pipeline', 'per-metric pass/fail gate', 'model registry'), reinforced by the keyword metadata, with no generic jargon-only phrasing.

3 / 3

Distinctiveness Conflict Risk

Scoped to a narrow niche — candidate vs. production pairwise comparison with per-metric tolerances and segment gating — with distinct triggers unlikely to fire for adjacent skills like deepchecks-tests or evidently-monitoring.

3 / 3

Total

12

/

12

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

metadata_version

'metadata.version' is missing

Warning

Total

15

/

16

Passed

Reviewed

Table of Contents