CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

auditing-deid-leakage

Adversarially scan already-de-identified clinical text for residual identifiers and emit a leakage report that blocks release on any hit. Use after OpenMed de-identification when the user asks to verify a redaction, prove no PHI/PII leaked, gate a dataset before sharing, or run a second-pass detector. Covers format and checksum detectors (SSN, Luhn for card numbers, MRN/account patterns, emails, phones, dates), entropy heuristics for high-randomness tokens, severity scoring, and a hard block-on-leak rule. This is the verification half of OpenMed's leakage-first ethos. Hand-off: re-run openmed.extract_pii on the de-id output and diff against expectations. License-free, local-first. Pairs after deidentifying-clinical-text.

74

Quality

93%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

82%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A strong, action-first body: executable quick-start code with a real checksum implementation, a clear sequenced workflow with an explicit release-blocking validation gate, and genuinely useful edge-case guidance (never log leaked plaintext, surrogate false positives, Luhn gating). The only deductions are minor: some rhetorical framing that echoes the description, an only-implied re-redact/re-scan recovery loop, and no use of reference files despite a body long enough to benefit from them.

Suggestions

Trim the rhetorical framing (the 'verified, not assumed' opening and the 'leakage-first ethos' sentence, which repeats the description) to push conciseness toward lean.

Make the recovery loop explicit in the Workflow: after 'locate and re-redact', add a step to re-run the deterministic scan + second-pass detector until findings are empty before re-attempting the release gate.

Consider moving the detector regex table and/or the standards links into a references/ file (e.g. references/detectors.md) so SKILL.md stays a leaner overview with well-signaled one-level-deep references.

DimensionReasoningScore

Conciseness

The body is dense and mostly earns its tokens — working quick-start code, a tight 5-step workflow, and high-value gotchas ("Never log the leaked value", "Checksum-gate card numbers", "Surrogates are not leaks"). A few rhetorical sentences could be trimmed without loss ("De-identification is verified, not assumed" framing, "It is the verification half of OpenMed's leakage-first ethos — gate on leakage, not on F1", which repeats the description), so it lands on anchor 4 (efficient, minor over-explanation) rather than anchor 5 (every token earns its place). It is clearly not anchor 3: nothing explains concepts Claude already knows.

4 / 5

Actionability

The quick start is fully executable, not pseudocode: a real Luhn implementation, a concrete DETECTORS regex dict, an actual API call (`openmed.extract_pii(deid_text)`) with entity handling, and a hard gate (`assert not leaked, "Release BLOCKED..."`). It is copy-paste ready and covers the common case end-to-end, matching anchor 5 rather than anchor 4 ('minor gaps').

5 / 5

Workflow Clarity

The 5-step workflow is clearly sequenced with an explicit validation checkpoint — step 5 "Block on any leak. The gate is binary for release: if findings is non-empty at high/critical, fail the export" — so the batch-operation cap of 3 does not apply. It falls just short of anchor 5 because the error-recovery loop is only implied ("so a reviewer can locate and re-redact") rather than an explicit re-redact-then-re-scan-until-clean step, and it is well above anchor 3, whose validation is missing or implicit.

4 / 5

Progressive Disclosure

No bundle files exist (no references/, scripts/, or assets/), and none are referenced from the body, so there are no broken or buried reference paths. The ~130-line body is well-sectioned (When to use, Quick start, Workflow, Hand-off, Edge cases, Standards) with everything reasonably kept inline for a single-purpose skill and external links grouped in a dedicated Standards section. This sits at anchor 4 (good structure, content appropriately placed) rather than anchor 5 because there are no well-signaled one-level-deep references at all, and at ~130 lines a detector-pattern table or standards list could plausibly live in a reference file.

4 / 5

Total

17

/

20

Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

An exemplary description: concrete and comprehensive on capabilities, explicit on both what and when, rich in natural trigger phrases users would actually say, and clearly disambiguated from the sibling de-identification skill it pairs with. Third-person voice is used consistently.

DimensionReasoningScore

Specificity

The description lists multiple concrete actions — "Adversarially scan already-de-identified clinical text for residual identifiers and emit a leakage report that blocks release on any hit" plus "format and checksum detectors (SSN, Luhn for card numbers, MRN/account patterns, emails, phones, dates), entropy heuristics... severity scoring, and a hard block-on-leak rule" — with comprehensive capability coverage. It exceeds anchor 4 ('minor gaps in coverage') because every core capability of the skill is named explicitly, so it is not the level below.

5 / 5

Completeness

Both questions are answered explicitly: what — "scan already-de-identified clinical text for residual identifiers and emit a leakage report that blocks release on any hit" with a Covers-list of detector families; when — "Use after OpenMed de-identification when the user asks to verify a redaction, prove no PHI/PII leaked, gate a dataset before sharing, or run a second-pass detector." This matches anchor 5 (explicit what AND when with concrete trigger phrases) and clearly exceeds anchor 4, whose 'when' is weaker ('Use when working with PDF files').

5 / 5

Trigger Term Quality

Natural user phrasings are comprehensively covered: "verify a redaction", "prove no PHI/PII leaked", "gate a dataset before sharing", "run a second-pass detector", with synonyms (PHI/PII, redaction, leak, gate, de-identification). A user needing this skill would naturally say several of these; anchor 4 ('a few natural terms missing') does not fit because both the synonyms and the colloquial trigger phrases are present.

5 / 5

Distinctiveness Conflict Risk

It carves out a clear niche — auditing the *output* of de-identification rather than de-identifying — and explicitly disambiguates from the sibling skill: "Pairs after deidentifying-clinical-text" and "This is the verification half of OpenMed's leakage-first ethos." The trigger conditions (verify/gate/second-pass on already-de-id text) would not plausibly fire the de-id skill itself, so conflict risk is minimal (anchor 5).

5 / 5

Total

20

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

maziyarpanahi/openmed

Path

skills/auditing-deid-leakage/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.