CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

testland/model-risk-evidence-matrix

Use when an ML model release is up for promotion and someone must decide which fairness and explainability evidence is required, or check the evidence against the model's risk tier. Assigns a low, medium or high tier, lists required artifacts and returns promote, needs-work or block.

Quality

84%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

73%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured decision-matrix skill with an explicit, checklist-driven gate workflow, concrete thresholds and rules, and properly split one-level-deep references. Its main weakness is token efficiency: the ~340-line body spends significant space on verbatim legal/literature quotations and textbook metric definitions that could be condensed or moved to references.

Suggestions

Trim the extended verbatim quotations (GDPR Article 22, Kleinberg/Chouldechova, EEOC Q&A) to one-line citations with the operative claim, cutting roughly a third of the prose while keeping every rule intact.

Move the Step 3 metric-definition table and formal expectation conditions into a reference file (e.g. references/fairness-metrics.md), keeping only the 'what it equalizes' column and the DPD difference-vs-ratio warning inline.

Add a short executable snippet (or a reference section) showing how to compute demographic_parity_difference with Fairlearn's MetricFrame, since the body gates on DPD values but never shows how to produce them.

DimensionReasoningScore

Conciseness

The body is mostly efficient (tables, thresholds, rules with little padding), but at ~340 lines it includes unnecessary explanation Claude already knows: re-derived definitions of demographic parity/equalized odds with formal conditions, and extended verbatim quotations (GDPR Article 22, Kleinberg/Chouldechova excerpts, EEOC Q&A) that could be tightened to short citations. More than 'minor instances of over-explanation', so below the 4 anchor.

3 / 5

Actionability

Concrete and executable throughout: exact bands ('DPD <= 0.05', '0.05 < DPD <= 0.10', 'DPD > 0.10'), a runnable command ('jq '.sensitive_features' model_card.json'), a four-field waiver format, and precisely triggered rules R1-R6. Minor gap: the body never shows how to compute DPD or run a MetricFrame, and that how-to is not in the bundle either, which keeps it below the 5 anchor.

4 / 5

Workflow Clarity

Steps 1-9 are clearly sequenced and Step 9 is an explicit 8-point gate checklist with per-step rule checks, refuse-to-promote conditions, and a validation safeguard ('Gate against the re-derived tier, never the declared one'). This is an assessment skill, so no fix-retry loop is required; the checklist-plus-explicit-checks structure matches the 5 anchor.

5 / 5

Progressive Disclosure

Two well-signaled, one-level-deep references ([references/alibi-explainability.md], [references/worked-example.md]) both exist and contain no nested references. However, the SKILL.md body itself carries substantial detail - metric definitions with formal conditions and extended legal citation prose - that could partly live in reference files, a minor organization gap that keeps it below the 5 anchor.

4 / 5

Total

16

/

20

Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it pairs an explicit, concrete 'Use when' trigger with specific, third-person statements of what the skill does and what verdicts it returns. Keyword coverage is good though not exhaustive of common synonyms like 'bias' or 'fairness audit'.

DimensionReasoningScore

Specificity

Lists several concrete actions in third person - 'Assigns a low, medium or high tier', 'lists required artifacts', 'returns promote, needs-work or block' - tied to a clear domain. It falls short of the 5 anchor because coverage has gaps (no mention of fairness metric checks, intersectional analysis, or vulnerability scan triage), and it is clearly above the 3 anchor's '1-2 concrete actions'.

4 / 5

Completeness

Explicitly answers both questions: the 'what' ('Assigns a low, medium or high tier, lists required artifacts and returns promote, needs-work or block') and a concrete 'when' ('Use when an ML model release is up for promotion and someone must decide which fairness and explainability evidence is required, or check the evidence against the model's risk tier'). The 'when' is explicit and specific, matching the 5 anchor rather than the 4 anchor where the trigger 'could be more explicit'.

5 / 5

Trigger Term Quality

Natural phrases a user would say include 'ML model release is up for promotion', 'fairness and explainability evidence', and 'risk tier'. Not a 5 because common synonyms such as 'bias check', 'fairness audit', or 'model card review' are absent; well above the 3 anchor since multiple natural trigger terms are present.

4 / 5

Distinctiveness Conflict Risk

A clear niche - promotion-gating of ML model releases on fairness/explainability evidence with named verdicts - that no generic skill would trigger on. Distinct triggers and outputs give minimal conflict risk, matching the 5 anchor.

5 / 5

Total

18

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

On this page