CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

testland/evidently-monitoring

Use for any question or task about data drift or quality monitoring for a production ML model, including a fired drift alert and metrics such as precision that degrade slowly unnoticed: Evidently reports with tests for data drift, target drift and performance regression as a CI gate or scheduled job, and a playbook to pick rollback, retrain, quarantine or alert re-tuning.

Quality

87%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

81%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A high-quality, actionable body: executable code at every step, explicit CI gating with validation and error-recovery loops, and genuinely non-obvious domain knowledge (data-definition inference pitfalls, business thresholds, alert triage discipline). The main improvements are consistency (Dataset vs raw DataFrame in run() calls), showing the per-column drift tuning the anti-patterns table demands, and splitting the alert-triage playbook into a reference file.

Suggestions

Reconcile the Dataset/DataFrame inconsistency: Step 2 says to pass `Dataset` objects to `report.run()`, but the code in Steps 3, 4, and 6 calls `report.run(reference_data=reference_df, current_data=current_df)` with raw DataFrames — pick one pattern and use it in every example.

Add a short code snippet showing per-column drift method/threshold configuration (e.g., PSI per column), since Step 4 and the anti-patterns table both instruct users to tune it but no example shows how.

Move the Step 7 triage playbook (signal table, hypothesis ranking, triage discipline) and the worked example into a `references/triage.md` file so the always-loaded SKILL.md stays lean and the detail loads only when an alert fires.

DimensionReasoningScore

Conciseness

The body is efficient and code-forward, and concentrates on non-obvious knowledge Claude lacks (ID-column inference pitfalls, probas_threshold vs the 0.5 default, label maturation windows). Minor over-explanation keeps it from 5: the opening marketing quote ("an open-source Python library with over 40+ million downloads... 100+ evaluation metrics") and download-count stats add nothing actionable.

4 / 5

Actionability

Nearly all guidance is executable copy-paste code (install, Dataset/DataDefinition, gated Report with FAIL/ERROR handling, BinaryClassification mapping, daily job). Two minor gaps: Step 2 instructs passing `Dataset` objects to `report.run()` while Steps 3, 4, and 6 pass raw DataFrames (`reference_df`/`current_df`), and per-column drift method/threshold tuning ("Configure the method and threshold per column on the preset") is described but never shown in code.

4 / 5

Workflow Clarity

Seven numbered steps are cross-linked with explicit "(Step N)" anchors, validation is explicit (gate on any FAIL/ERROR test, `raise SystemExit`), feedback loops exist (fix the pipeline then "re-run the report before returning the model to live traffic"; the worked example ends with "the re-run passes"), and the triage-discipline bullets ("Never retrain before ruling out hypotheses 1-3") act as a checklist. This matches the 5 anchor's explicit validation, error-recovery loops, and checklists.

5 / 5

Progressive Disclosure

Well-organized single-file skill with clear section headers, a numbered "How to use" summary, and a References section that annotates each external link's scope — all references are one level deep. Minor organization gap: the ~60-line Step 7 alert-triage playbook and the worked example are inlined in a ~330-line file though they are only needed when an alert fires, making them candidates for a reference file; this fits 4's "minor organization gaps" rather than 3's inlined-content-that-clearly-belongs-elsewhere.

4 / 5

Total

17

/

20

Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description that explicitly states both what the skill does and when to use it, with concrete trigger phrases (fired drift alert, slowly degrading precision) and specific capabilities (CI gate, scheduled job, rollback/retrain/quarantine playbook). Trigger-term coverage and distinctiveness are good rather than exhaustive, with a few missing synonyms and a broad "any question or task" claim creating minor conflict risk.

DimensionReasoningScore

Specificity

"Evidently reports with tests for data drift, target drift and performance regression as a CI gate or scheduled job, and a playbook to pick rollback, retrain, quarantine or alert re-tuning" lists multiple specific concrete actions covering detection, gating, scheduling, and remediation. Matches the comprehensive-coverage anchor; score 4 would require notable coverage gaps, which are absent.

5 / 5

Completeness

Explicitly answers "when" ("Use for any question or task about data drift or quality monitoring for a production ML model, including a fired drift alert and metrics such as precision that degrade slowly unnoticed") and "what" (drift/target/performance-regression report tests as a CI gate or scheduled job plus a remediation playbook) with concrete trigger phrases, matching the 5 anchor exactly.

5 / 5

Trigger Term Quality

Natural trigger phrases are present ("data drift", "quality monitoring", "a fired drift alert", "metrics such as precision that degrade slowly"), but common variations users would say — "concept drift", "model performance", "retraining", "MLOps monitoring" — are missing. Good coverage with a few natural terms missing fits the 4 anchor better than comprehensive 5 or partial 3.

4 / 5

Distinctiveness Conflict Risk

The description pins a clear niche (Evidently-based production drift monitoring with named triggers like a fired drift alert), but the claim "Use for any question or task about data drift or quality monitoring" is broad and could overlap with a generic model-monitoring or alternate drift-tooling skill. Minor overlap risk with closely related skills fits 4 rather than the minimal-conflict 5.

4 / 5

Total

18

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

On this page