Content
71%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
A highly actionable, well-sequenced skill body with excellent copy-paste examples covering all advertised workflows. Its weaknesses are moderate redundancy and time-sensitive version detail in the main body, and a merged-in DeepSpeed sub-bundle whose second-level references are deeply nested and poorly labeled, undermining navigation despite the clean top-level structure.
Suggestions
Drop the duplicated conversion example — keep either the Quick start diff or Workflow 1's original-vs-converted scripts, not both — and remove filler lines (the `accelerate config` question list, "Used by: ...") to tighten the body.
Move version-specific details ("Version: 1.11.0+", "FSDP: PyTorch 1.12+") into a clearly labeled version-compatibility note so they don't go stale in the main flow.
Fix the second-level navigation in references/deepspeed/overview.md: replace placeholder descriptions ("08.md - 08 documentation") with real topic summaries, split the 454KB tutorials.md into focused files, or surface key DeepSpeed content directly in the overview so nothing important is buried two levels deep.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The body is mostly lean code-and-commands, but it includes unnecessary material: the 4-line conversion example is shown in full twice (the diff in Quick start, then again as Workflow 1's original-vs-converted scripts, ~40 duplicated lines), and filler like the "Questions" list from `accelerate config` and Resources lines such as "Tutorial: 'Accelerate your scripts'" and "Used by: ..." add tokens without instruction value. Time-sensitive version numbers ("Version: 1.11.0+", "FSDP: PyTorch 1.12+") are also present outside any deprecated/version-note section, which the guidelines penalize. | 3 / 5 |
Actionability | Nearly every section is copy-paste-ready: the numbered 4-line conversion, per-setup launch commands (single/multi-GPU/multi-node with exact flags), mixed-precision constructor args, a complete deepspeed_config.json, the FSDP plugin setup, `accelerator.accumulate`, `save_state`/`load_state` guarded by `is_main_process`, and `set_seed`. The only placeholder is `dataset`, which is inherent to training examples; the examples cover all the common cases the description advertises. | 5 / 5 |
Workflow Clarity | Workflows are clearly sequenced — the conversion steps are literally numbered (+1..+4), each workflow follows an enable → prepare → launch pattern, and the "Common issues" section gives error-recovery fixes (wrong device placement, accumulation not working, FSDP seed divergence). Not anchor 5 because there are no explicit validation checkpoints (e.g., verifying the setup runs on one GPU before scaling to multi-node, or `accelerate test`), leaving checkpoints implicit rather than built into the workflows. | 4 / 5 |
Progressive Disclosure | The SKILL.md body itself is a well-signaled overview with one-level references (megatron-integration.md, custom-plugins.md, performance.md, and the merged-tool guides all exist and are linked with descriptive tables). However, scoring against the actual bundle structure: the merged DeepSpeed guide nests a second reference level (references/deepspeed/overview.md → references/deepspeed/references/*.md, including a 454KB tutorials.md monolith), and its "Reference Files" index signals them with useless placeholders like "08.md - 08 documentation" and "other.md - Other documentation", so the deep material is effectively buried two levels down. | 3 / 5 |
Total | 15 / 20 Passed |