gating-deid-leakage

Add a CI gate that fails the build when an OpenMed de-identification model's recall on a held-out PHI set drops below threshold or any critical identifier leaks. Use when the user wants a pytest test or CLI step that exits nonzero on de-id regression, wants to wire OpenMed's leakage-first release gates into GitHub Actions / CI, needs a recall floor plus zero-leakage assertion against a synthetic held-out set, or wants to block merges that weaken de-identification. Trigger on "CI gate", "fail the build", "regression test", "de-id recall threshold", "block the merge", "exit nonzero", or "leakage check in CI" for OpenMed.

Quality

94%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable skill body with executable pytest/CLI/CI examples and a validated, fail-closed workflow. The only weakness is minor rhetorical padding in the intro and a few asides that assume less competence than the rest of the doc.

Suggestions

Trim the rhetorical opener ('Logs, baselines, and models drift. The only durable defense ...') to a one-line purpose statement; the 'When to use' section already motivates the skill.

Move the HIPAA/NIST/Actions standards list into a short references file or collapse to bare links so the body stays focused on the gate mechanics.

Consider extracting the full G1a–G8 gate semantics into a referenced file and keeping only the CI-wiring essentials inline, since the skill already defers to evaluating-with-leakage-gates for those.

Dimension	Reasoning	Score
Conciseness	Mostly lean code-and-steps content, but the opener ('Logs, baselines, and models drift. The only durable defense ...') and a few ethos lines ('a passing-but-not-required gate protects nothing') are mild over-explanation that could be trimmed.	4 / 5
Actionability	Two complete, copy-paste-ready code blocks (pytest fixture + tests, GitHub Actions YAML) and a fully flagged CLI invocation cover the common cases with executable detail.	5 / 5
Workflow Clarity	A six-step workflow with explicit validation checkpoints (assert critical_leakage_count == 0 and recall ≥ floor in step 4, 'make it required' in step 5) and a feedback path (--issue-on-failure), consistent with the fail-closed gating discipline.	5 / 5
Progressive Disclosure	Well-organized into clearly signaled sections with one-level-deep pointers to related skills (evaluating-with-leakage-gates, building-gold-corpus) and external standards; no bundle files exist, and the inline content is appropriately scoped, though a couple of reference-heavy sections could live in separate files.	4 / 5
	Total	18 / 20 Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, third-person description that concretely states the capability, gives explicit 'Use when' and 'Trigger on' guidance with natural phrases, and occupies a clearly distinct niche. No vague fluff or over-claims.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions — 'fails the build', 'exits nonzero on de-id regression', 'wire ... release gates into GitHub Actions / CI', 'block merges that weaken de-identification' — with comprehensive coverage of the gating task.	5 / 5
Completeness	Explicitly answers both what ('Add a CI gate that fails the build when ... recall ... drops below threshold or any critical identifier leaks') and when (a 'Use when ...' clause plus a 'Trigger on ...' clause with concrete triggers).	5 / 5
Trigger Term Quality	An explicit 'Trigger on' clause supplies natural phrases users would say — 'CI gate', 'fail the build', 'regression test', 'de-id recall threshold', 'block the merge', 'exit nonzero', 'leakage check in CI' — including synonyms.	5 / 5
Distinctiveness Conflict Risk	Scoped to a narrow niche (OpenMed de-identification CI gating with recall floor + zero-leakage assertion) with distinct, domain-specific triggers, so overlap with unrelated skills is minimal.	5 / 5
	Total	20 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: maziyarpanahi/openmed
Path: skills/gating-deid-leakage/SKILL.md
Commit: 80da98c

Reviewed: about 23 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.