Detect Skill Duplicates

Identify duplicate and overlapping skills in a discovery.json. Output conforms to duplicates.schema.json.

Hybrid phase: deterministic pre-screen (Jaccard similarity) → LLM judgement (one subagent per candidate pair, written to its own file) → deterministic clustering.

Three contracts are validated at the IO boundary:

• Prep input (discovery.json) and prep output (<prompts-dir>/index.json) — see duplicates-prompts-index.schema.json
• Each subagent verdict (<verdicts-dir>/<idx>.json) — see duplicate-verdict.schema.json. Per-pair verdicts validate non-strictly: a malformed verdict from one subagent is recorded in metadata.failed_pairs[] and the run continues.
• Final duplicates.json output — see duplicates.schema.json. Malformed output exits with code 2.

Inputs

• A discovery.json produced by discover-skills.
• Optional --max-pairs cap (default 10) and --allow-cross-repo (default off).

Step 1: Prepare candidate-pair prompts

python3 <skill-dir>/scripts/prepare_duplicates.py \
  --discovery "$DISCOVERY_PATH" \
  --prompts-dir "$PROMPTS_DIR"   # default: <dirname(discovery)>/duplicates-prompts

Output:

<prompts-dir>/
├── index.json    ← manifest: {metadata, items: [{idx, pair_id, skill_a, skill_b, similarity_score, prompt_filename, verdict_filename}]}
├── 000.txt       ← judgement prompt for pair 0 (skill A vs skill B)
├── 001.txt       ← judgement prompt for pair 1
└── ...

The prep script:

1. Reads every skill from discovery.skills[]
2. Computes Jaccard similarity over tokenised name + description (and over body_preview)
3. Generates candidate pairs (within-repo by default; cross-repo with --allow-cross-repo)
4. Filters never-compare pairs (same content_hash, fixture paths, identical skill_id)
5. Caps at --max-pairs (default 10), preferring pairs where both skills are agent-loaded and have higher similarity
6. For each candidate, loads both skills' SKILL.md content and writes the judgement prompt to <idx>.txt
7. Writes index.json linking each idx → pair_id → prompt path → expected verdict path

Step 2: Dispatch one subagent per pair, all in parallel

For each items[i] in index.json, dispatch a subagent. Default --max-pairs is 10, so this is a single batch — issue ALL of them in one assistant message via parallel Task tool calls. Don't sub-batch ("first 10, then next 10") — there's nothing to batch.

Pass each subagent:

• The path to its prompt file (e.g. <prompts-dir>/000.txt)
• The path it should write its verdict to (e.g. <verdicts-dir>/000.json)

The subagent must:

1. Read the prompt
2. Reason through the comparison (duplicate / overlapping / independent)
3. Write the verdict to the verdicts path:

{
  "pair_id": "p001",
  "verdict": "duplicate" | "overlapping" | "independent",
  "reason": "<one-sentence>",
  "dominant": "<skill_id_a | skill_id_b | null>"
}

Failure handling: if a subagent fails to write or writes malformed JSON, retry that one. If still malformed, leave the file missing — finalize records it in metadata.failed_pairs[].

Verdicts directory (default <dirname(discovery)>/duplicates-verdicts/) — pre-create:

mkdir -p "$VERDICTS_DIR"

Step 3: Cluster + finalize

python3 <skill-dir>/scripts/finalize_duplicates.py \
  --prompts "$PROMPTS_DIR" \
  --verdicts "$VERDICTS_DIR" \
  --output "$OUTPUT_PATH"        # default: <dirname>/duplicates.json

The finalize script:

1. Reads index.json from the prompts directory
2. Reads each <idx>.json from the verdicts directory
3. Builds an undirected graph from duplicate verdicts
4. Computes connected components (transitive clusters)
5. Picks dominant_skill_id per cluster (most-voted-dominant → has-evals → most lines → alphabetical)
6. Computes severity per cluster (size, harness overlap)
7. Records overlapping_pairs from overlapping verdicts
8. Computes estate summary (cluster count, reduction potential)
9. Writes duplicates.json
10. Cleans up the prompts and verdicts directories. Pass --keep-intermediates if you want them preserved.

Verify

jq -e '.schema_version == "1.0"' "$OUTPUT_PATH" > /dev/null

Sanity-check: every skill_id mentioned in clusters[].skill_ids should exist in the source discovery.json.

Standalone testability

• Prep alone: see candidate pairs without LLM cost.
• Mock verdicts: write JSON files into duplicates-verdicts/<idx>.json; run finalize to test clustering / severity.
• Per-pair retry: bad verdict → re-run just that idx; finalize picks up the new file.