Content
73%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
Well-structured content with a strong worked example (input config, mapper prompt, output matrix, action items, trend) and an explicit verification loop that avoids the hallucination-trust anti-pattern. The main weakness is actionability: the central script is partly pseudocode with undefined helper functions and no config wiring, so it cannot actually produce the promised coverage matrix.
Suggestions
Make scripts/ai-coverage.py fully executable: define read(), extract_acs(), and read_all() (e.g., glob + regex extraction per the ac_extraction pattern) and wire the YAML config keys (spec_path, test_globs, ac_extraction.pattern) into the script via CLI args or a config loader.
Add the output-parsing/formatting step that turns the raw LLM response into the structured coverage matrix (per-AC tier, covering test names, action items, trend vs. prior run) shown in the worked example, or state explicitly that Claude must render the matrix from the raw response.
Trim duplication: the 'When to use' section restates the frontmatter description, and the Limitations/'reads code, doesn't run it' point plus the anti-patterns verification rows repeat guidance already covered in references/continuous-coverage-and-verification.md.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The body is dense and example-driven with no conceptual padding Claude already knows — the Overview's line-vs-spec-section distinction is skill-specific framing, and the worked example demonstrates the deliverable rather than explaining trivia. Not 5: "When to use" repeats the frontmatter description almost verbatim, and Limitations ("The LLM reads code; doesn't run it") plus the anti-patterns table duplicate verification guidance already detailed in the reference file; not 3 because these are minor trim opportunities, not several padded sections. | 4 / 5 |
Actionability | Concrete elements exist — the YAML config ("pattern: 'AC-(\\d+\\.\\d+):'"), a real openai.chat.completions call with a complete system prompt, and a fully specified output format — but the script is partially pseudocode: "spec_text = read(spec_path)", "ac_list = extract_acs(spec_text)", and "read_all(test_globs)" are undefined, the YAML config keys are never wired into the script, and "print(response.choices[0].message.content)" only dumps the raw response, so the code cannot produce the structured matrix shown. Not 4: these are missing key details rather than minor gaps; not 2 because the LLM-call core, prompt, and input/output formats are genuinely concrete. | 3 / 5 |
Workflow Clarity | "How to use" gives a clear five-step sequence ("Point the mapper at the spec doc + the test globs" → "Run the LLM mapper" → "Read the coverage matrix" → "File one action item per gap" → "Schedule it weekly and verify the LLM's claims") with explicit validation for this batch operation, a feedback loop via the trend comparison ("Compare with prior run"), and an anti-patterns checklist ("Spot-check + cross-reference every high-priority AC"). Not 4: validation checkpoints are explicitly present inline and elaborated one level deep in the reference, satisfying the top anchor. | 5 / 5 |
Progressive Disclosure | The bundle has a single, real, one-level-deep reference (references/continuous-coverage-and-verification.md) that is clearly signaled in three places with a description of its contents ("weekly CI scheduling and how to verify the LLM's coverage claims"), and the SKILL.md appropriately keeps the methodology and worked example inline while deferring cadence/verification detail. Not 5: the body's code comment "# scripts/ai-coverage.py" and the reference's CI step "run: python scripts/ai-coverage.py" both invoke a scripts/ path that does not exist in the bundle — a minor organization gap in the file structure. | 4 / 5 |
Total | 16 / 20 Passed |