CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

testland/model-performance-regression-gate

Use when a retrained ML model is a candidate for promotion and CI must block it if it got worse. Compares held-out accuracy, F1, AUC or RMSE with the production model within a tolerance, and checks key segments with Deepchecks so a global gain cannot hide a slice regression.

Quality

92%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A high-quality, highly actionable skill body: an 8-step CI-gate workflow with fully executable code at every stage, explicit validation checkpoints and non-zero exits, error-triage artifacts, an anti-patterns table, and an honest limitations section. Detail is appropriately pushed to a single real reference file. The main trim opportunities are the duplicated intro/neighbor-differentiation prose and the Deepchecks internals explanation.

DimensionReasoningScore

Conciseness

The body is code-dense and efficient with no library-intro filler, but includes minor over-explanation that could be trimmed: the four bullets on WeakSegmentsPerformance internal mechanics, the intro/"Differentiation from neighbors" section that repeats the frontmatter description, and the redundant `Dataset` re-import in Step 6. It fits the level-4 anchor (efficient, minor instances of over-explanation) rather than level 5, where every token would earn its place.

4 / 5

Actionability

Every step ships fully executable, copy-paste-ready code or config: pip install commands, joblib artifact loading, sklearn metric computation for both classification and regression variants, a tolerance gate with sys.exit codes, Deepchecks checks with explicit conditions, and a complete GitHub Actions job. This matches the level-5 anchor; the only nit (`predict_proba[:, 1]` assumes binary while the text notes multiclass args) does not undermine executability for the common case.

5 / 5

Workflow Clarity

Eight clearly sequenced steps each carry explicit validation: per-metric PASS/FAIL printing, failure accumulation with non-zero exit, passed_conditions() gating, report HTML saved and uploaded `if: always()` for triage, plus an anti-patterns checklist and a Limitations section covering edge cases (small test sets, models without predict_proba). This is not a destructive or batch operation, so the cap does not apply, and the content matches the level-5 anchor.

5 / 5

Progressive Disclosure

Good structure: the tunables and full metric signatures are correctly split into a real, one-level-deep reference (references/gate-config-and-metrics.md, verified to exist), clearly linked with markdown links at the points of use (Steps 3 and 8), with no nested references. It sits at level 4 rather than 5 because the bulk of the content remains inline (~300 lines) and the trailing References section duplicates the inline link definitions — minor organization gaps rather than a lean overview pointing to well-signaled materials.

4 / 5

Total

18

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it states an explicit, concrete trigger condition ("Use when a retrained ML model is a candidate for promotion and CI must block it") and enumerates specific actions covering global metrics, tolerances, and per-segment checks. Third person, concise, and clearly differentiated from neighboring skills. Only minor gap is a few missing synonyms (e.g., "model comparison", "champion/challenger").

DimensionReasoningScore

Specificity

The description lists multiple concrete, specific actions — "Compares held-out accuracy, F1, AUC or RMSE with the production model within a tolerance", "checks key segments with Deepchecks", and blocking promotion in CI — with comprehensive coverage of the gate's behavior. It fits the level-5 anchor (multiple specific concrete actions, comprehensive coverage) rather than level 4, since no meaningful capability of the niche is left unnamed.

5 / 5

Completeness

It explicitly answers both questions: "Use when a retrained ML model is a candidate for promotion and CI must block it if it got worse" gives a concrete 'when' with trigger phrasing, and the following sentence gives a clear, multi-part 'what'. This matches the level-5 anchor; the 'when' is explicit and specific, not the weaker level-4 form.

5 / 5

Trigger Term Quality

Good natural keyword coverage: "retrained ML model", "candidate for promotion", "CI must block it", "accuracy, F1, AUC or RMSE", "production model", "slice regression". A few natural terms users might say are missing (e.g., "model comparison", "champion/challenger", "model validation"), so it does not reach the level-5 comprehensiveness anchor.

4 / 5

Distinctiveness Conflict Risk

The niche — pairwise candidate-vs-production metric comparison as a promotion gate at deploy time — is clearly distinct from generic model evaluation or drift monitoring, with minimal trigger overlap. The Deepchecks mention is incidental to the dominant promotion-gating trigger, so conflict risk with a general Deepchecks skill is minimal, matching the level-5 anchor.

5 / 5

Total

19

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

On this page