Content
63%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
The body is a genuinely actionable, well-sequenced guide whose domain knowledge (study-design requirements, layered search strategy, fitness evaluation, honesty rules) is high-value. Its weaknesses are token efficiency — the pandas/statsmodels cookbook re-teaches basics — and the absence of any reference-file split for the heavy inline catalogs.
Suggestions
Trim the Data Loading Cookbook to the non-obvious patterns (URL download with XPT/format detection, REST API pagination) and drop the basic pandas loader list and Merge/Clean/Analyze block, which Claude can already write unaided.
Move the Layer-2 repository/tool catalog and the loading cookbook into clearly signaled one-level-deep reference files (e.g., references/repositories.md, references/loaders.md), keeping SKILL.md as an overview.
Add explicit validation checkpoints with feedback loops in Step 4 — e.g., after loading, verify row counts and that required variables exist, and retry with a different parser or inspect the raw bytes on failure.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The domain-specific guidance (study-design mapping, search layers, fitness checklist, honesty rules) is lean, but roughly 70 lines of the Data Loading Cookbook and Merge/Clean/Analyze sections re-teach basics Claude already knows ("df = pd.read_csv", "model = smf.ols(...)", the eight-format pandas loader list). That is more than minor over-explanation, placing it at anchor 3 rather than 4. | 3 / 5 |
Actionability | Concrete executable code (URL download with format detection, REST API pagination, regression) and specific tool names ("re3data_search_repositories", "DbGaP_search_studies", "Synapse_search_entities") with decision rules ("Does X predict CHANGES in Y over time?" → longitudinal). Minor gaps: placeholder URLs and variable names ("api_url", "exposure_var") rather than a worked end-to-end example, so it falls just short of anchor 5. | 4 / 5 |
Workflow Clarity | A clear 5-step sequence (Understand → Search → Evaluate → Download/Analyze → Report) with checklists in Steps 3 and 5 and some checkpoints ("verify the codebook", "When unsure of the format, download a small sample first and inspect"). No explicit validate→fix→retry feedback loops exist, keeping it below anchor 5. | 4 / 5 |
Progressive Disclosure | The body is well-sectioned with clear headers, but it is a single ~180-line file with the repository catalog (Layer 2) and data-loading cookbook inlined — content that belongs in separate reference files. This matches anchor 3 ("content that should be separate is inline"); it is above anchor 2 because structure and navigation are genuinely good. | 3 / 5 |
Total | 14 / 20 Passed |