CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/evidently-monitoring

Use Evidently OSS (100+ evaluation metrics, declarative testing API) to detect data drift, target drift, and model-performance regression, wired into CI as a gate (a Report run with include_tests) and into production monitoring as a continuous check; reports as HTML + JSON for both human review and pipeline assertions. Includes a drift-alert triage playbook: classify the fired alert's signal, rank root-cause hypotheses (upstream schema change, pipeline bug, training-serving skew, seasonality, genuine population shift), and pick rollback, retrain, quarantine, or alert re-tuning. Use when you need a drift or quality gate, a scheduled monitoring job, or a structured triage of a fired drift alert, for a tabular ML model. Built on the Evidently API specifically: for DeepChecks-based validation suites use deepchecks-tests instead.

69

Quality

87%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

81%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-built skill body: executable code for every phase (install, drift report, CI gate, performance presets, scheduled monitoring), valuable corrections of outdated Evidently API assumptions, and an exemplary triage workflow with validation gates and feedback loops. The main weaknesses are minor: some duplicated/padded content, a couple of non-executable placeholder functions, and no code example for the per-column threshold tuning it repeatedly recommends.

DimensionReasoningScore

Conciseness

The body is largely efficient, executable guidance that adds non-obvious API knowledge ("there is no separate TestSuite class... pass include_tests=True", ".dict() exposes top-level 'metrics' and 'tests' only - there is NO top-level 'status' key"), with only minor trimmable instances: the marketing quote "an open-source Python library with over 40+ million downloads" and the Worked example duplicating the Step 4 gate code nearly verbatim. Matches the level-4 anchor (efficient, minor over-explanation); not 5 because those padding/redundancy instances exist, not 3 because the bulk of tokens are actionable rather than explanatory.

4 / 5

Actionability

Mostly copy-paste-ready code: `pip install evidently`, `Report([DataDriftPreset()])` with `run(reference_data=, current_data=)`, `include_tests=True`, the `result["tests"]` FAIL/ERROR gate with `raise SystemExit`, and the regression/classification presets. Minor gaps keep it below the fully-executable level-5 anchor: Step 6 depends on undefined environment helpers (`load_production_window`, `load_reference_window`, `notify_oncall`), Step 5 uses undefined `ref`/`cur` variables inconsistent with the rest, and per-column drift method/threshold tuning is recommended twice ("Tune the per-column drift method (psi, etc.)", anti-patterns table) but never shown in code.

4 / 5

Workflow Clarity

Clear numbered sequence ("How to use" items 1-7 mapped to Steps 1-7) with explicit validation checkpoints and feedback loops: the CI gate raises SystemExit on any FAIL/ERROR, triage mandates "re-run the report before returning the model to live traffic", "Never retrain before ruling out hypotheses 1-3", and the worked example closes the loop ("widens that column's threshold... and the re-run passes"). The triage-discipline bullets and signal/hypothesis tables act as checklists, matching the level-5 anchor; validation is present, so no cap applies.

5 / 5

Progressive Disclosure

No bundle files exist (references/, scripts/, assets/ are absent), so all content lives in one ~270-line SKILL.md. It is well-sectioned (When to use, Steps 1-7, Worked example, Anti-patterns, Limitations, References) with clearly labeled one-level-deep external doc links ([Evidently docs], [Evidently Report], [Evidently drift preset]). This is good structure with minor gaps — the level-4 anchor — rather than level 5, since the self-contained triage playbook and worked example are inlined where a references/ split would keep the main body leaner.

4 / 5

Total

17

/

20

Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it names concrete capabilities across CI gating, production monitoring, and alert triage, gives explicit 'Use when' triggers scoped to tabular ML models, and actively disambiguates from the adjacent deepchecks-tests skill. Its only deductions are the second-person voice penalty on specificity and a few missing natural trigger synonyms.

DimensionReasoningScore

Specificity

The description lists many concrete, comprehensive actions — "detect data drift, target drift, and model-performance regression, wired into CI as a gate", "classify the fired alert's signal, rank root-cause hypotheses", "pick rollback, retrain, quarantine, or alert re-tuning" — which otherwise matches the level-5 anchor. However, it is written in second-person/imperative voice ("Use Evidently OSS...", "Use when you need..."), which the judging guidelines penalize by reducing the specificity score by 1. Not a 3: it lists far more than 1-2 concrete actions with comprehensive coverage.

4 / 5

Completeness

Both 'what' and 'when' are explicit and concrete: the what is "detect data drift, target drift, and model-performance regression... wired into CI as a gate... and into production monitoring" plus the triage playbook, and the when is "Use when you need a drift or quality gate, a scheduled monitoring job, or a structured triage of a fired drift alert, for a tabular ML model." This directly matches the level-5 anchor (explicit what AND when with concrete trigger phrases); the explicit 'Use when' clause means no cap applies.

5 / 5

Trigger Term Quality

Good keyword coverage with natural phrases users would actually say: "data drift", "drift or quality gate", "scheduled monitoring job", "structured triage of a fired drift alert", "tabular ML model", "production monitoring". A few common natural terms are missing as literal phrases (e.g., "model monitoring", "drift detection" as a noun phrase, "data quality"), so it falls just short of the comprehensive-with-synonyms level-5 anchor; it clearly exceeds the 'some relevant keywords' level-3 anchor.

4 / 5

Distinctiveness Conflict Risk

Clear niche (Evidently OSS for tabular ML drift monitoring/gating/triage) with distinct triggers and explicit disambiguation against the nearest competing skill: "Built on the Evidently API specifically: for DeepChecks-based validation suites use deepchecks-tests instead." Minimal conflict risk; matches the level-5 anchor and is clearly above the 'minor overlap risk' level-4 anchor.

5 / 5

Total

18

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

Table of Contents