batch-processing-clinical-text

Run large-scale batch NER, PII extraction, or de-identification over many clinical notes on-device with OpenMed, with sharding, checkpointing, resumability, and append-only JSONL output. Use when the user needs to process a corpus or folder of notes, de-identify a dataset, run NER over thousands of documents, build a resumable batch pipeline, or stream results to JSONL without holding everything in memory. Covers process_batch / BatchProcessor / BatchItem / BatchResult, the operation= selector (analyze_text | extract_pii | deidentify), iter_process streaming, the PHI-safe on_progress callback, chunking long documents, and no-PHI logging. Produces a resumable batch runner over an OpenMed model.

Quality

92%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable with executable code and a well-sequenced, validated batch workflow including resume and failure-reconciliation loops. It is mostly lean, though some streaming content is repeated across sections and the dense API reference could be offloaded to a bundled reference file.

Suggestions

De-duplicate the iter_process/streaming explanation: keep the executable detail in one section and reference it from the Workflow steps instead of restating it.

Move the full process_batch / BatchProcessor / BatchItemResult signature dump into a references/ file (e.g. API.md) and keep only the one-line usage in SKILL.md to improve progressive disclosure.

Tighten the intro paragraph, which restates the description's sharding/checkpointing/resumability phrasing almost verbatim.

Dimension	Reasoning	Score
Conciseness	Efficient and free of over-explanation of known concepts, but the streaming/iter_process behavior is described in both the 'Streaming + PHI-safe progress' section and the 'Workflow'/'Resumable batch runner' sections, leaving minor repetition that could be tightened.	4 / 5
Actionability	Provides fully executable, copy-paste-ready code for process_batch, BatchProcessor configuration, iter_process streaming, and a complete run_resumable function with real file I/O and checkpoint logic, covering the common NER/PII/de-id cases.	5 / 5
Workflow Clarity	The 7-step Workflow is clearly sequenced with explicit feedback loops for a batch operation — resume by skipping ids already in the append-only JSONL, continue_on_error isolation, and a reconcile step via get_failed_results() — so validation/recovery is not missing.	5 / 5
Progressive Disclosure	Well-organized into clearly headed sections with one-level external links (jsonlines.org, HIPAA, source path) and no nested references, but the full API signature dump and the resumable runner are inlined in a single dense file rather than split into a reference file.	4 / 5
	Total	18 / 20 Passed

Description

96%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, concrete description that clearly states capabilities and gives explicit, natural trigger phrases for when to use the skill. The only soft spot is mild overlap with single-note de-identification skills, which the batch-scale triggers mostly resolve.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions — 'batch NER, PII extraction, or de-identification' plus 'sharding, checkpointing, resumability, and append-only JSONL output' — giving comprehensive coverage rather than generic verbs.	5 / 5
Completeness	Explicitly answers both what ('Run large-scale batch NER, PII extraction, or de-identification… with sharding, checkpointing, resumability') and when ('Use when the user needs to process a corpus or folder of notes…') with concrete trigger phrases.	5 / 5
Trigger Term Quality	The 'Use when' clause covers natural phrases users would say — 'process a corpus or folder of notes', 'de-identify a dataset', 'run NER over thousands of documents', 'build a resumable batch pipeline', 'stream results to JSONL' — with good synonym coverage.	5 / 5
Distinctiveness Conflict Risk	The batch/corpus scope ('thousands of documents', 'resumable batch pipeline') carves a clear niche, but 'de-identify a dataset' overlaps with adjacent single-note de-identification skills, leaving minor conflict risk.	4 / 5
	Total	19 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: maziyarpanahi/openmed
Path: skills/batch-processing-clinical-text/SKILL.md
Commit: 80da98c

Reviewed: about 24 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.