benchmark-pii-recall

Benchmark an OpenMed PII model with synthetic gold spans and report label-aware exact-span and grapheme recall without emitting identifier surfaces. Use when an agent must compare a model, threshold, backend, or quantized artifact and enforce a recall floor before release.

Quality

87%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

82%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A tight, actionable body anchored by a runnable benchmark and explicit release gates, with clean sectioning and a single external reference. It is executable and well-structured, with only minor room to tighten rationale and make the retry loop explicit.

Suggestions

Make the failure/retry feedback loop explicit in the procedure (e.g., 'If the recall floor is not met, adjust threshold or fixtures and re-run') rather than relying on the assert alone.

Trim the justification sentence in the intro or fold it into the release-gates section to tighten token efficiency.

Confirm the ../../examples/v16_policy_audit_release_gates.py link resolves from the skill location so the single reference is not a dead path.

Dimension	Reasoning	Score
Conciseness	Efficient and mostly lean — short intro, numbered procedure, and an executable code block that earns its place — with only minor rationale ('A missed direct identifier is a privacy failure...') that could be trimmed.	4 / 5
Actionability	Provides a complete, copy-paste-ready Python benchmark including the install command, fixtures with offsets, scoring calls, and an assertion enforcing the recall floor.	5 / 5
Workflow Clarity	The six-step procedure plus the in-code assert and 'Release gates' section give a clear sequence with explicit validation, though the fix-and-retry feedback loop is implied rather than spelled out.	4 / 5
Progressive Disclosure	Well-organized into Procedure, Runnable benchmark, Release gates, and Repository example, with one clearly signaled one-level-deep reference; no bundle files exist to verify further structure.	4 / 5
	Total	17 / 20 Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A precise, third-person description that clearly states capabilities and an explicit 'Use when' trigger with concrete release-gate language. It is highly specific and distinct, with only minor room for broader natural-language synonyms.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions — 'Benchmark an OpenMed PII model with synthetic gold spans', 'report label-aware exact-span and grapheme recall', 'without emitting identifier surfaces' — giving comprehensive coverage of what the skill does.	5 / 5
Completeness	Explicitly answers both what (benchmark PII recall, report exact-span and grapheme recall, avoid emitting surfaces) and when ('Use when an agent must compare a model, threshold, backend, or quantized artifact and enforce a recall floor before release').	5 / 5
Trigger Term Quality	Good keyword coverage ('compare a model, threshold, backend, or quantized artifact', 'recall floor before release'), but the terms are domain-technical and lack common synonyms a non-expert user might say.	4 / 5
Distinctiveness Conflict Risk	Targets a narrow niche — OpenMed PII recall benchmarking with a release recall floor — with distinct triggers and minimal overlap risk against other skills.	5 / 5
	Total	19 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
relative_links	Relative link issues: 1 suspicious	Warning

	Total	15 / 16 Passed

Repository: maziyarpanahi/openmed
Path: skills/benchmark-pii-recall/SKILL.md
Commit: b161a18

Reviewed: 2 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.