Content
92%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
An exemplary operational skill body: dense, fully executable, with a validated dry-run-then-submit workflow and a smart pattern of deferring field-level detail to CLI help lookups. Its one real defect is that the twice-referenced docker-images.md is absent from the bundle, breaking the progressive-disclosure path for the custom-image flow.
Suggestions
Add the missing docker-images.md file to the bundle (e.g., under references/), or if it cannot ship, inline the essential Dockerfile/push steps and drop the dangling links in the 'Starting a run' pre-flight step and the 'Custom Docker images' section.
Add a one-line validation cue before `databricks air cancel ... --all` (e.g., run `databricks air list` first and confirm the runs about to be cancelled with the user), since it destructively kills every active run.
Update the two docker-images.md links to the final bundle-relative path (e.g., references/docker-images.md) once the file exists, so the reference resolves correctly in the packaged skill.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | Lean and dense throughout: command tables, an env-var table, a launcher-shapes table, and a consistent "the CLI's built-in help is the source of truth... Run `databricks air run -h config.<field>`" pattern that defers exhaustive field lists to runtime instead of inlining them. Not 4: there is no padded conceptual explanation — even version-sensitive details (CLI 1.20.0, nested snapshot format) are quarantined in a fallback subsection with a lookup command, and "Today's accelerator types are `GPU_1xA10` and `GPU_8xH100`; confirm with..." immediately de-risks the time-sensitive claim. | 5 / 5 |
Actionability | Fully executable, copy-paste-ready guidance covers the common cases: session setup commands, a complete workload YAML, `databricks air run --file workload.yaml --dry-run -o json -p <profile>`, secrets scope/key creation commands, a per-framework `command:` table (torchrun/accelerate/deepspeed/composer), and logs flags (--node/--tail/--minutes/--download-to). Not 4: the examples are complete and runnable as written, including the Python snippet for reading `$HYPERPARAMETERS_PATH`. | 5 / 5 |
Workflow Clarity | "Starting a run" is a clearly numbered 5-step sequence with explicit validation and feedback loops: pre-flight resource checks (step 3), a dry-run with "A config-validation error is a hard stop: ... show that message to the user and fix the YAML" (step 4), and idempotency-key discipline on submit (step 5); failure handling closes the loop (pull logs, surface traceback, let the user decide). Not 4: checkpoints are explicit with a fix-and-retry path, not merely implied — the peripheral `cancel --all` command appears only in a reference table, not as a walked-through workflow. | 5 / 5 |
Progressive Disclosure | The body itself is well structured (clear overview of three flows, Docker detail correctly split out, one-level references clearly signaled as "Read [docker-images.md](docker-images.md)"), but the actual bundle contains no docker-images.md — no references/, scripts/, or such file exists anywhere in the skill directory — so both references (in the pre-flight step and the Custom Docker images section) are dangling and the "step-by-step guidance" they promise is unreachable. Not 4: a broken primary reference is more than a minor organization gap; not 2: the in-body structure and signaling are good and nothing is improperly inlined. | 3 / 5 |
Total | 18 / 20 Passed |