CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

testland/evidently-monitoring

Use when a tabular ML model needs a drift or quality gate in CI, a scheduled monitoring job, or triage of a fired drift alert. Uses Evidently reports with tests for data drift, target drift and performance regression, and a playbook to pick rollback, retrain, quarantine or alert re-tuning.

Quality

89%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

81%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A strong operational skill: executable code at every step, current-API gotchas documented in comments, validation-gated CI patterns, and a genuinely expert triage playbook with anti-patterns and limitations. Weaknesses are minor - a small amount of promotional/padded prose, placeholder functions in the scheduling step, and a triage section long enough to warrant its own reference file.

Suggestions

Drop the marketing quotation ('an open-source Python library with over 40+ million downloads...') from the intro; Claude needs no persuading, and it saves tokens (conciseness).

Replace or footnote the undefined 'load_production_window()' and 'notify_oncall()' calls in Step 6 with concrete stubs (e.g., a pandas read of a date-filtered window) so the scheduling example is fully runnable (actionability).

Move the Step 7 triage playbook (signal table, five ranked hypotheses, discipline rules) into a references/triage.md file and keep a short decision summary inline, reducing SKILL.md body size (progressive_disclosure).

DimensionReasoningScore

Conciseness

Mostly efficient, code-first content whose comments carry non-obvious API facts ('there is NO top-level "status" key. Gate on any test that did not pass'). Minor over-explanation could be trimmed: the marketing quote 'an open-source Python library with over 40+ million downloads' and 'The standard pattern compares two datasets' add nothing Claude needs. Fits anchor 4 (minor instances of over-explanation) rather than 3, since the excess is isolated, not pervasive.

4 / 5

Actionability

Steps 1-5 and the worked example are copy-paste ready with real imports, file names, and a concrete CI failure path ('raise SystemExit(f"Evidently drift gate failed: {len(failed)} test(s)")'). However, Step 6 relies on undefined placeholders ('load_production_window', 'notify_oncall') rather than executable code, a minor gap that keeps it below anchor 5.

4 / 5

Workflow Clarity

Clear numbered sequence (Steps 1-7) with a cross-referenced 'How to use' summary, explicit validation checkpoints (gating on 'FAIL'/'ERROR' test status, 'raise SystemExit'), a feedback loop (worked example: two columns drift -> widen threshold or retrain -> 'the re-run passes'), and a triage checklist with hard rules ('Never retrain before ruling out hypotheses 1-3'). Destructive remediation actions (rollback, quarantine, retrain) are all guarded by validation discipline, so no cap applies.

5 / 5

Progressive Disclosure

Well-organized single-file skill: clear section headers, clearly signaled one-level references to canonical Evidently docs (bracketed names resolved in a References section, e.g. '[Evidently Report]: https://docs.evidentlyai.com/docs/library/report'). The inlined ~60-line triage playbook (signal table, five ranked hypotheses, discipline rules) is content that could live in a separate reference file, which is the minor organization gap of anchor 4 rather than the mis-signaling of anchor 3.

4 / 5

Total

17

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: third-person voice, explicit 'Use when' triggers covering three distinct use cases, and a concrete capability list naming the library, the test types, and the remediation playbook. The only weakness is slightly thin synonym coverage for monitoring-related phrasings users might naturally use.

DimensionReasoningScore

Specificity

Lists multiple concrete actions covering the full workflow: 'drift or quality gate in CI, a scheduled monitoring job, or triage of a fired drift alert', plus 'tests for data drift, target drift and performance regression' and a 'playbook to pick rollback, retrain, quarantine or alert re-tuning'. Coverage is comprehensive with no gaps.

5 / 5

Completeness

Explicitly opens with 'Use when a tabular ML model needs a drift or quality gate in CI, a scheduled monitoring job, or triage of a fired drift alert' and separately states the what: 'Uses Evidently reports with tests... and a playbook to pick rollback, retrain, quarantine or alert re-tuning'. Both what and when are answered clearly with concrete trigger phrases.

5 / 5

Trigger Term Quality

Good natural-term coverage ('drift', 'quality gate', 'CI', 'monitoring job', 'drift alert', 'performance regression', 'retrain') that users would plausibly say, but misses common variations like 'model monitoring', 'ML monitoring', or 'model performance'. Fits anchor 4 (a few natural terms missing) rather than 5's comprehensive synonym coverage.

4 / 5

Distinctiveness Conflict Risk

Clear niche: Evidently-based drift gating, monitoring, and alert triage for tabular ML models. The named library plus drift-specific triggers (drift gate, drift alert triage) make it unlikely to fire for unrelated skills.

5 / 5

Total

19

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

On this page