Content
77%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
An expert-grade operational skill: the two-kernel architecture, image-reference rule, timeout model, and error catalog are all rendered as concrete executable guidance with strong validation loops. Its weaknesses are a ~780-line monolithic body that keeps deep-dive reference material inline despite pointing to env-setup.md elsewhere, and noticeable repetition of the timeout, notification-payload, and image-ref rules.
Suggestions
Move the reference-heavy material — the error-kind catalog under "When the job fails", the network-egress debugging guide, and "Common env gotchas" — into a separate reference file (like the existing env-setup.md pattern), leaving SKILL.md as a lean overview with clearly signaled one-level-deep pointers.
Deduplicate the repeated rules: state the container-vs-job timeout distinction, the compute_done payload fields, and the im-… image-reference rule once each (the image rule alone appears in prose, bold text, two code comments, and its own section), and cut the rest to a back-reference.
Consider relocating the GPU tier table and measured latency table to a reference file as well, keeping only the headline numbers (~3–6 min first GPU result, ~5–10 s warm resubmit) inline where the workflow needs them.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The body is dense with host-specific operational detail and never explains concepts Claude already knows, but it is noticeably repetitive: the container-timeout vs job-timeout explanation appears in both "Two timeouts, one timeline" and again in "Workflow"; the `compute_done` payload field list (status/exit_code/featured_files plus error_kind/system_hint/deadline_fired) is given twice; and the "pass `r['image']`, not the bare env name" rule is stated at least four times ("Carry `r['image']` … NOT the env name", "**Pass `r['image']` (the `im-…` ref)**", two code comments, plus the dedicated "image-reference rule" section). This is more than the minor trimmable instances of anchor 4, so anchor 3 fits. | 3 / 5 |
Actionability | Fully executable, copy-paste-ready code cells for both the `repl` and `compute_provider` kernels with real kwargs (image refs, gpu tiers, volumes, timeout_seconds), a per-error-kind → remedy catalog, measured latency and GPU-tier tables, and concrete teardown commands like `modal.Sandbox.list(app_id=app.app_id, tags={"claude-science-job": "<job_id>"})` — anchor 5. | 5 / 5 |
Workflow Clarity | The multi-step process is clearly sequenced (check the `compute_details` ledger via `spec_sha` → build env in the `compute_provider` kernel → hand off via a workspace JSON file → append the ledger block → submit → `wait_for_notification` → attach/close) with explicit validation checkpoints: `compute_provider_config()` staleness check "before anything expensive", per-error-kind recovery loops, and an end-of-task teardown checklist ("every handle closed … no sandbox of yours still alive"). Feedback loops for the risky batch/spend operations are present, matching anchor 5. | 5 / 5 |
Progressive Disclosure | Section headers are clear and `env-setup.md` is repeatedly and clearly signaled with its scope, but no bundle files exist (`references/`, `scripts/`, `assets/` are absent) and the ~780-line body inlines deep-dive material that belongs in separate files — the full error troubleshooting catalog, the egress-fence debugging guide, the env gotchas list, and the GPU-tier/latency tables. Structure is real but the monolithic inline bulk matches anchor 3 rather than the well-split content of anchor 4. | 3 / 5 |
Total | 16 / 20 Passed |