CtrlK
BlogDocsLog inGet started
Tessl Logo

auditing-subgroup-fairness

Audit an OpenMed NER or de-identification model for performance disparities across demographic subgroups (sex, age band, race/ethnicity when available) using openmed.eval.fairness_report. Use when the user wants per-subgroup recall and leakage, wants to check whether de-identification under-protects a group, wants to surface a documentation gap where subgroup data is missing, or needs equalized-odds-style disparity numbers for a clinical model. Trigger on "fairness", "subgroup", "bias audit", "disparity", "equalized odds", "under-protected group", "per-group recall", or "STANDING Together" for an OpenMed model.

80

Quality

100%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Low

Low-risk findings worth noting

SKILL.md
Quality
Evals
Security

Quality

Content

100%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is a tight, actionable overview: executable quick-start code, a clearly sequenced audit workflow with a verification gate, and well-signaled hand-offs to adjacent skills. It respects token budget while remaining concrete.

DimensionReasoningScore

Conciseness

Lean and information-dense; it assumes Claude's competence (no general explanations of NER, model cards, or equalized odds) and every section — intro, metrics, quick start, workflow, hand-offs, gotchas — earns its place.

5 / 5

Actionability

Quick start provides copy-paste-ready, fully executable code (import, fairness_report call with concrete args, result printing, and an assert gate) covering the common audit case.

5 / 5

Workflow Clarity

A clear six-step sequence with analytical checkpoints (read leakage first, compute disparity, locate worst_group, document the gap), an explicit assert verification gate, and a gotchas section that serves as a checklist; the operation is read-only so the destructive-cap does not apply.

5 / 5

Progressive Disclosure

Well-organized into clearly headed sections with one-level-deep, clearly signaled skill hand-offs (building-gold-corpus, evaluating-with-leakage-gates, authoring-model-cards, benchmarking-clinical-ner) and a source-of-truth pointer; no nested references and no bundle files needed.

5 / 5

Total

20

/

20

Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, complete description: it states concrete capabilities, gives an explicit and comprehensive trigger list, and carves out a distinct niche. Third-person voice is used throughout and no vague fluff is present.

DimensionReasoningScore

Specificity

Names the domain (OpenMed NER/de-identification) and multiple concrete actions — auditing performance disparities across subgroups, surfacing documentation gaps, producing equalized-odds disparity numbers — via a named tool (openmed.eval.fairness_report); third-person voice.

5 / 5

Completeness

Explicitly answers both what (audit subgroup performance disparities) and when (a multi-clause "Use when..." plus an explicit "Trigger on..." list with concrete trigger phrases).

5 / 5

Trigger Term Quality

Comprehensive natural trigger list including synonyms and a domain standard: "fairness", "subgroup", "bias audit", "disparity", "equalized odds", "under-protected group", "per-group recall", "STANDING Together".

5 / 5

Distinctiveness Conflict Risk

A clearly bounded niche — OpenMed fairness/de-id under-protection audits — with distinctive triggers, making conflict with unrelated skills unlikely.

5 / 5

Total

20

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository
maziyarpanahi/openmed
Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.