Content
85%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
This is a highly actionable, well-sequenced runbook with exceptional validation and failure-mode coverage, and it wastes almost no tokens on things Claude already knows. Its main structural weakness is that everything lives in one 580-line SKILL.md, forcing the full app/iOS/palette regression detail into context even for runs that will skip those steps.
Suggestions
Move steps 10b-10f (iOS simulator, dropdown overflow, logs pane, shortcut recorder, palette regressions) into separate reference files (e.g. references/ios.md, references/app-ui.md) and signal them from the body with one-line pointers, so only the relevant scenario is loaded when its permissions/conditions allow it.
De-duplicate the repeated blocks: the 'Cmd-comma does NOT work here' comment and the `open -g -n` env-override snippet each appear twice; state the convention once in the Conventions section and reference it from 10d/10e/10f.
Bundle or verify `build-report.py` (step 12 references `.claude/skills/integration-test/build-report.py`, which is outside this skill's directory) so the report generation step is self-contained with the skill.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The body is dense and almost every token is project-specific gotcha knowledge Claude could not know (port+1 TLS listener collision, stale CLI embed in the app bundle, defaults Data round-trip, pkill self-kill), so it earns its length. It is not a 5 because there is verbatim duplication that could be trimmed: the 'Cmd-comma does NOT work here' comment is repeated in full in steps 10d and 10e, the `open -g -n` env-override block is duplicated in 10f, and a few prose passages (10e recorder discussion) could be tightened. | 4 / 5 |
Actionability | Nearly every step is copy-paste executable: exact env vars, ports, token, curl commands with expected status codes (401/200), xcodebuild with explicit BUILD SUCCEEDED check, md5 stale-install verification, defaults export/import restore. The few prose-only parts (step 06, 10f keyboard scenarios) are conditional and explicitly justified as manual/permission-gated rather than pseudocode, which matches the anchor for fully executable guidance covering the common cases. | 5 / 5 |
Workflow Clarity | A clearly numbered 01-12 sequence with a 'read first' conventions section, per-step evidence specs (meta.json/output.txt/PNG), explicit pass/warn/fail verdict policy ('never abort the run'), and abundant validation checkpoints (healthz polling, 401/200 pair, spool replay poll, md5 match, appearance restore guarded by a known-original check, teardown with defaults import). This run touches user state (dark mode, defaults domain) and validation is present at every such point, so the destructive/batch cap does not apply. | 5 / 5 |
Progressive Disclosure | The single file is well-structured with headers, but it is a ~580-line monolith with zero reference files: the five app-UI regression scenarios (10b-10f, roughly 250 lines) are self-contained sub-procedures that clearly belong in separate reference files loaded only when those scenarios run. The body also points at `build-report.py`, which is not part of this bundle's structure, and no references/scripts/assets directories exist. | 3 / 5 |
Total | 17 / 20 Passed |