CtrlK
BlogDocsLog inGet started
Tessl Logo

building-gold-corpus

Scaffold a synthetic gold-standard annotation project for evaluating OpenMed NER and de-identification models — label schema, annotation guidelines, BRAT or Label Studio config, and disjoint train/dev/test splits. Use when the user wants to create eval fixtures, set up annotation, define a label set, write guidelines, configure an annotation tool, or build a held-out gold set for the OpenMed eval harness. Trigger on "gold corpus", "annotation project", "label schema", "annotation guidelines", "BRAT", "Label Studio", "train dev test split", or "build eval fixtures" for OpenMed. Committed gold must be synthetic; licensed (i2b2/n2c2/MIMIC) data is eval-only and never committed.

76

Quality

94%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable with executable examples, a disciplined multi-step workflow including validation, and a well-structured reference layout. It is efficient but carries a little framing prose that could be tightened.

Suggestions

Trim the opener 'You can't evaluate what you can't measure against' to enter the substantive guidance sooner and improve token efficiency.

Consider splitting the full label list and fixture-schema details into a references file once bundle dirs are introduced, keeping SKILL.md as a tighter overview.

Add an explicit 'validate -> fix -> re-validate' loop callout in step 7 to make the error-recovery feedback loop unmistakable.

DimensionReasoningScore

Conciseness

Lean and efficient; assumes Claude's competence and avoids explaining basics. Minor framing prose ('You can't evaluate what you can't measure against') could be trimmed, but overall every section earns its place.

4 / 5

Actionability

Provides a concrete JSON fixture shape, an executable load_fixtures snippet, a specific directory tree, and a 7-step workflow with named tools and config files — copy-paste ready and covering common cases.

5 / 5

Workflow Clarity

Clear 7-step sequence with explicit validation checkpoints (step 7 'Validate and commit: Run load_fixtures; confirm spans align and ids are unique') and an Edge cases section serving as a feedback loop for batch/destructive operations.

5 / 5

Progressive Disclosure

Well-organized into clearly signaled sections (Quick start, Workflow, Hand-off, Edge cases, Standards & references) with one-level-deep external references; no bundle files exist, so structure is self-contained but slightly reference-heavy inline.

4 / 5

Total

18

/

20

Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is exemplary: it names concrete capabilities, provides comprehensive natural trigger terms, and explicitly covers both what the skill does and when to use it within a distinct OpenMed niche.

DimensionReasoningScore

Specificity

Lists multiple concrete actions — 'label schema, annotation guidelines, BRAT or Label Studio config, and disjoint train/dev/test splits' — covering the project comprehensively.

5 / 5

Completeness

Explicitly answers 'what' (scaffold a synthetic gold-standard annotation project) and 'when' via a clear 'Use when...' clause plus a 'Trigger on...' list of concrete phrases.

5 / 5

Trigger Term Quality

Comprehensive natural triggers including synonyms and tool names ('gold corpus', 'annotation project', 'label schema', 'annotation guidelines', 'BRAT', 'Label Studio', 'train dev test split', 'build eval fixtures').

5 / 5

Distinctiveness Conflict Risk

A clear niche (OpenMed gold-corpus scaffolding) with distinct, tool- and domain-specific triggers; minimal overlap risk with other skills.

5 / 5

Total

20

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository
maziyarpanahi/openmed
Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.