Content
42%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
The body has good bones — clear use cases, an excellent honesty rule guarding against fabricated meta-analysis results, and a valuable domain-reasoning section — but it is bloated with textbook meta-analysis content Claude already knows, contains no executable code, and cites a script ('python_implementation.py') that is absent from the bundle. Roughly half the file could move to a reference document or be cut outright.
Suggestions
Cut or move to a reference file the textbook sections Claude already knows — the fixed/random-effects explanation, the I² interpretation ladder, 'Sources of Heterogeneity', 'Quality Tiers', and 'Limitations & Caveats' — keeping only skill-specific decision rules (which thresholds to apply, which model to pick when).
Fix the dangling reference: 'python_implementation.py' is cited as the meta-analysis implementation but no scripts/ directory exists in the bundle. Either ship the script or inline a working inverse-variance pooling code example so the honesty rule is actionable.
Convert at least one use-case workflow (e.g., the TCF7L2 meta-analysis) into a numbered, executable sequence with the concrete tool calls and a validation checkpoint (e.g., 'if matched associations lack beta + CI, report descriptive mode and stop').
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | Large stretches of the body are textbook material Claude already knows: fixed vs random-effects models, the I² interpretation ladder ('<25% low... >75% considerable'), 'Sources of Heterogeneity' (ancestry, phenotype, platform, winner's curse), generic quality tiers, and a long 'Limitations & Caveats' section. These padded sections crowd out the genuinely skill-specific guidance (the honesty rule, the domain-reasoning section) — matching the 'noticeably verbose; several unnecessary explanations or padded sections' anchor. | 2 / 5 |
Actionability | There is some concrete guidance — named tools like 'gwas_get_associations_for_snp' and 'OpenTargets_get_variant_credible_sets', and a precise honesty rule with concrete output fields — but the workflows are outline bullets ('Search for all T2D studies', 'Filter by sample size and ancestry') with no executable code anywhere in the body. Worse, the honesty rule anchors correctness to 'python_implementation.py parses these from the GWAS Catalog beta/or_value + range fields', yet no scripts/ directory or such file exists in the bundle, so the referenced implementation cannot actually be used. | 3 / 5 |
Workflow Clarity | The four use-case workflows do list a rough sequence (search → filter → extract → assess) with expected outcomes, and the honesty rule supplies one implicit checkpoint (fall back to 'method=descriptive' when effect sizes are unusable). But the sequences are unnumbered bullet sketches with no explicit validation checkpoints for batch operations like cross-study association retrieval or replication-rate calculation, matching the 'steps listed but validation gaps' anchor. | 3 / 5 |
Progressive Disclosure | Section headers and structure are present (Overview, Use Cases, Statistical Methods, Tools Used, Glossary), so it is not unstructured. However, the bundle contains no references/, scripts/, or assets/ directories — everything, including ~150 lines of statistical-methods and quality-tier reference material that belongs in a separate file, is inlined — and the single file the body does cite ('python_implementation.py') does not exist, so the one navigation pointer is broken. | 3 / 5 |
Total | 11 / 20 Passed |