Content
92%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
A well-engineered skill body: fully executable command coverage for all five subcommands, a correctly ordered workflow with an explicit pre-flight validation gate and error-recovery guidance, and clean deferral of detail to verified one-level-deep references. The only weakness is minor redundancy — two load-bearing warnings are each stated twice, and the citation boilerplate adds length without task value.
Suggestions
State the out-of-vocabulary warning and the ~1,000-sample threshold once (workflow step 2 and 'When to use' respectively) and have the caveats section cross-reference them rather than repeat them.
Move the 'Citing Scientific Agent Skills' and 'Upstream' blocks into a reference file or the frontmatter metadata to keep the body focused on task execution.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The body is dense with non-obvious, load-bearing detail (gated-repo 401/403 semantics, `trust_remote_code` revision pinning, defaults worth overriding with exact values, split-leakage warnings) and assumes Claude's competence throughout — no generic explanations of transformers, parquet, or CLIs. It falls short of anchor 5 only because a few warnings are stated twice (out-of-vocabulary dropping appears in workflow step 2 and again in caveats; the ~1,000-sample threshold appears in 'When to use' and in caveats) and the citation/administrative sections ('Citing Scientific Agent Skills', 'Upstream') are boilerplate rather than task-serving — minor trimming is possible, which is anchor 4, not noticeably-verbose anchor 3. | 4 / 5 |
Actionability | Every workflow step is a complete, copy-paste-ready command with flags and realistic arguments (`waypoint embed --model outpost-bio/Waypoint-6m --data dataset.parquet --output embeddings.parquet`), including both classification and regression finetune invocations, converter calls for three profiler formats, and the coverage-check script. Specific examples cover the common cases and concrete defaults are given ('warmup_steps: 1000' → ~50, hidden dims 256/512/768 per model), matching anchor 5 exactly. | 5 / 5 |
Workflow Clarity | The workflow is six clearly numbered, correctly sequenced steps with an explicit validation checkpoint placed before anything else ('Check vocabulary coverage before anything else', with a concrete 0.8 threshold and instruction to re-examine taxonomy labels before trusting downstream numbers), error-recovery guidance (a 401/403 'almost always means access was never requested on that specific repo'), and a load-bearing caveats checklist for the fragile operations (split leakage, batch effects, small-sample regime). This matches anchor 5 ('Clear sequence with explicit validation steps; feedback loops for error recovery; checklists for complex processes') and exceeds anchor 4, whose checkpoints lack the feedback-loop framing. | 5 / 5 |
Progressive Disclosure | SKILL.md is a true overview: each workflow step keeps only the essential command and key gotchas inline, while bulk detail is split across four real, substantive reference files (cli-reference.md, compass-benchmark.md, data-preparation.md, python-api.md — all verified present in the bundle) plus two scripts, each listed with a one-line description in dedicated References/Scripts sections. References are one level deep (reference files point only to bundled scripts, not to further .md files), navigation is easy, and nothing that belongs in a separate file is inlined — matching anchor 5. | 5 / 5 |
Total | 19 / 20 Passed |