Content
75%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
Highly actionable content with executable calculator commands, concrete templates, and explicit checklists, supported by a well-structured bundle of two real reference files and a working script. The main weakness is token efficiency: several sections teach concepts Claude already knows (peeking, p-values, client/server-side basics) and the Common Mistakes section duplicates Core Principles, while the long body inlines content that could be pushed to references.
Suggestions
Trim or remove sections that re-teach known concepts — the 'Statistical Significance' definition (p < 0.05), 'The Peeking Problem' explanation, and the client-side/server-side implementation basics — leaving only the skill-specific prescriptive guidance (don't peek, pre-commit to sample size).
Cut the 'Common Mistakes' section, which restates 'Core Principles' (hypothesis first, single variable, don't stop early), or collapse it into a single short list of analysis-phase-only mistakes like cherry-picking segments.
Move the Quick Reference sample-size table and the Test Types / Implementation sections into references/sample-size-guide.md or a new reference file, keeping the SKILL.md body as a lean overview that points to the calculator command and the references.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The body re-explains concepts Claude already knows — "95% confidence = p-value < 0.05; Means <5% chance result is random", the peeking problem definition, and client-side vs server-side implementation basics — and "Common Mistakes" largely restates "Core Principles". It is mostly efficient (tight bullets, tables, task-specific guidance dominates), so it sits at anchor 3 rather than the noticeably-verbose anchor 2, but the several known-concept explanations keep it below 4. | 3 / 5 |
Actionability | Copy-paste-ready commands that match the bundled script's real interface ("python3 scripts/sample_size_calculator.py --baseline 0.05 --mde 0.20 --json"), a fill-in hypothesis template with weak/strong examples, concrete metric tables, and explicit checklists cover the common cases. This matches anchor 5's fully executable, copy-paste-ready standard for an instruction-type skill. | 5 / 5 |
Workflow Clarity | The sequence (assessment → hypothesis → sizing → metrics → variants → launch → analysis) is clear via section order and backed by an explicit pre-launch checklist and a numbered analysis checklist. Not 5 because the ordering is implicit rather than an explicit procedure and there are no error-recovery/feedback loops; not 3 because validation checkpoints are explicit and concrete. | 4 / 5 |
Progressive Disclosure | Two real, clearly signaled one-level-deep references ("For detailed sample size tables and duration calculations: See references/sample-size-guide.md", "For templates: See references/test-templates.md") plus a real script, verified to exist in the bundle. Not 5 because the ~300-line body inlines encyclopedic content (test-type definitions, implementation tool lists, common mistakes) that belongs in reference files; not 3 because the references that do exist are well signaled and appropriately split. | 4 / 5 |
Total | 16 / 20 Passed |