Content
71%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
The body is highly actionable with concrete, executable commands for every environment and a well-sequenced workflow that includes a mandatory resource guard and launch verification. Its weaknesses are redundancy (the GPU check is repeated three times), the absence of failure-recovery guidance after launch verification, and a monolithic single-file structure where the W&B guide and CLAUDE.md example could be split into reference files.
Suggestions
Consolidate the GPU-availability check: state the nvidia-smi/MPS commands once (e.g. in Step 0) and have Step 2 reference it instead of repeating them, removing the duplicated free-GPU criteria.
Add an error-recovery branch to Step 5: instruct what to do when screen -ls shows no session or the process exited early (e.g. tail the tee'd log, report the error, do not retry blindly).
Move the W&B integration code block and the CLAUDE.md configuration example into a reference file (e.g. references/wandb.md, references/config-examples.md) and link to them from the main body to reduce inline bulk.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The body is mostly command-driven and tight, but the nvidia-smi GPU-availability check appears nearly verbatim three times (Step 0 inline check, Step 2 pre-flight, and the Local deploy section), and passages like "Benefits: version-tracked, multi-server sync with one push, no rsync include/exclude rules needed" explain things Claude already knows. This fits the 'mostly efficient but could be tightened' anchor rather than the minor-trim level of 4. | 3 / 5 |
Actionability | Nearly every step ships copy-paste-ready commands with explicit placeholders: the rsync include/exclude filters, the ssh + screen + conda launch command, "vastai destroy instance <INSTANCE_ID>", "modal run modal_launcher.py", and "vastai ssh-url <INSTANCE_ID>". Common cases (local, remote, Vast.ai, Modal) are each covered with concrete examples, matching the fully-executable top anchor. | 5 / 5 |
Workflow Clarity | Steps 0-7 are clearly sequenced with real checkpoints: the mandatory Step 0 guard ("STOP IMMEDIATELY... Do NOT fabricate"), Step 5 "Verify Launch" via screen -ls / modal app list, and Step 7's download-results-before-destroy ordering for the destructive vast.ai teardown. It falls short of 5 because there is no error-recovery loop after verification — no guidance for what to do if the screen session is absent or the launch fails. | 4 / 5 |
Progressive Disclosure | The skill is a single ~300-line file with no bundle files (no references/, scripts/, or assets/ exist), and substantial content that could live in separate reference files is inlined — the full W&B integration code block (Step 3.5), the large CLAUDE.md example block, and the Modal/Vast.ai setup notes. Section structure is good, but content that should be split out is inline, which is the score-3 anchor rather than the well-split structure of 4. | 3 / 5 |
Total | 15 / 20 Passed |