Content
53%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
The body is well structured as an overview with a real, correctly-shaped bundle, but the executable guidance is incomplete: the flagship annotate example has a hidden prerequisite (clinvar.vcf.gz / --clinvar), bulk retrieval has no command, and no workflow includes validation. Minor redundancy between Key Features and Implementation Details adds avoidable tokens.
Suggestions
Make the annotate example runnable as printed: document the ClinVar VCF prerequisite, show the '--clinvar' flag, and add a one-line command for fetching the ClinVar VCF from the FTP URL already in references/api_reference.md.
Replace the prose-only bulk download section ('Supports obtaining ClinVar datasets via FTP') with an executable command (e.g., wget from ftp://ftp.ncbi.nlm.nih.gov/pub/clinvar/vcf_GRCh38/) and link references/api_reference.md at that point.
Trim the hedging sentences ('Output is expected to include...', 'Designed for pipeline use: ...') and the Key Features / Implementation Details overlap, and add a validation checkpoint after annotation (e.g., verify output with 'bcftools view -h annotated.vcf').
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The body is mostly list-based with no conceptual padding, but includes trimmable hedging ('Output is expected to include matching ClinVar records/identifiers', 'Designed for pipeline use: deterministic input/output files', 'Recommended when you need reproducible, high-throughput annotation') and the 'Key Features' section substantially restates 'Implementation Details' ('ClinVar search via NCBI E-utilities' appears in both). This matches anchor 3 — mostly efficient but could be tightened — rather than anchor 4, where over-explanation is only a minor instance. | 3 / 5 |
Actionability | Two copy-paste-ready commands with real query syntax are given ('python scripts/search.py --term "BRCA1[gene] AND pathogenic[CLNSIG]"', 'python scripts/annotate.py --input input.vcf --output annotated.vcf'), but key details are missing: the annotate command silently requires a 'clinvar.vcf.gz' file and an undocumented '--clinvar' flag (scripts/annotate.py defaults to 'clinvar.vcf.gz'), bulk download has no command or URL despite being a key feature, and the interpretation workflow stops at 'see references/clinical_significance.md' with no follow-up step. This fits anchor 3 — some concrete guidance but incomplete, missing key details — better than anchor 4's 'minor gaps'. | 3 / 5 |
Workflow Clarity | Rough sequences exist (search → interpret; download → annotate) via per-feature example commands, but there are no explicit steps connecting them and no validation checkpoints — nothing tells the user to verify the annotated VCF (e.g., with bcftools) or how to obtain the ClinVar VCF prerequisite. Per the rubric's cap, batch operations (whole-VCF annotation, bulk downloads) without validation steps cannot score above 3, and the implicit-checkpoint profile matches anchor 3 exactly. | 3 / 5 |
Progressive Disclosure | The body is a genuine overview (~60 lines) pointing to a real, well-organized bundle: both referenced files exist and are one level deep ('references/api_reference.md', 'references/clinical_significance.md'), scripts are referenced by path, and the reference files contain no nested pointers. It falls short of anchor 5 on a minor organization gap: the bulk-download section mentions FTP but never signals that the FTP URLs and directory layout live in references/api_reference.md, and the reference pointers appear only under 'Key Features' rather than at the point of need. | 4 / 5 |
Total | 13 / 20 Passed |