auditing-subgroup-fairness
Audit an OpenMed NER or de-identification model for performance disparities across demographic subgroups (sex, age band, race/ethnicity when available) using openmed.eval.fairness_report. Use when the user wants per-subgroup recall and leakage, wants to check whether de-identification under-protects a group, wants to surface a documentation gap where subgroup data is missing, or needs equalized-odds-style disparity numbers for a clinical model. Trigger on "fairness", "subgroup", "bias audit", "disparity", "equalized odds", "under-protected group", "per-group recall", or "STANDING Together" for an OpenMed model.
80
100%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
View guide
Low
Low-risk findings worth noting
Low
Low-risk findings.
1 low severity finding. Worth noting, but not necessarily harmful.
W011: Third-party content exposure detected (indirect prompt injection risk).
What this means
The skill exposes the agent to untrusted, user-generated content from public third-party sources, creating a risk of indirect prompt injection. This includes browsing arbitrary URLs, reading social media posts or forum comments, and analyzing content from unknown websites.
Why it was flagged
The required workflow (“Run `fairness_report` on the model + suite” and “Tag the gold corpus by group”) consumes the provided gold fixtures/corpus text (PHI spans and their `metadata`), which are outsider-authored when the operating user did not author/select the suite/fixtures; this free-form corpus text is then ingested into the LLM context via the evaluation/inference path.
- Repository
- maziyarpanahi/openmed
- Commit
80da98c
- Audited
- Security analysis
Is this your skill?
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.