Content
63%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
A highly actionable, information-dense skill: every section carries repo-specific commands, file paths, and CI job names that Claude could not guess, and the flaky-test probe technique is documented with an executable example and clear guardrails. The weaknesses are structural: the sharding/WSL CI-infrastructure detail is inlined at length rather than split into a reference file, and the main sharding section is an unbroken wall-of-text paragraph.
Suggestions
Move the PR CI sharding topology and the three WSL coupling bullets (version pinning, prepared distribution caching) into a references/ file (e.g. references/ci-sharding.md), keeping a 3-5 line summary plus a clearly signaled link in SKILL.md — this would raise both conciseness and progressive_disclosure.
Break the ~250-word 'Sharding in PR CI' run-on paragraph into a short table (workflow → jobs → check names) or bulleted list so the shard counts per platform are scannable.
Add the PowerShell probe variant as an actual snippet (or a one-level reference) instead of prose description, making all platform variants of the flakiness loop copy-paste ready.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | Almost everything is repo-specific detail Claude cannot infer (CI job names, artifact naming, shard mechanics) with no padding about known concepts, but the ~250-word sharding run-on paragraph and the WSL version-pinning/VHD-caching subsections are lengthy content that could be tightened or trimmed to a summary. This is more than the 'minor instances' of anchor 4, but nowhere near the padding of anchor 2. | 3 / 5 |
Actionability | Concrete, copy-paste-ready guidance throughout — 'pnpm run smoketest -g "<suite name>" --tracing', a complete YAML probe step, 'gh run download <run-id> -n logs-... -D ./logs', and the az CLI call with real org/project values. Minor gaps keep it below 5: the PowerShell and AzDO probe variants are described in prose ('checks $LASTEXITCODE after each run') rather than shown. | 4 / 5 |
Workflow Clarity | The flaky-test probe workflow is clearly sequenced with checkpoints (where to add it, fail-fast to preserve traces, 'Remove the loop before merging'), and the debugging section gives explicit prioritization ('Check these first for a hang') plus the exit-code pass/fail signal. It falls short of anchor 5 because the artifact-inspection flow lacks an explicit validate/fix/retry loop, though the before-hook note partially covers log-missing cases. | 4 / 5 |
Progressive Disclosure | Headers are clear and cross-links to sibling skills (unit-tests, code-oss-logs, azure-pipelines) are well signaled, but the skill is ~160 lines with no bundle files, and content that clearly belongs one level deep — the PR CI sharding job topology and the WSL provisioning/cache detail — is fully inlined in SKILL.md. This matches anchor 3 ('content that should be separate is inline') rather than anchor 4's 'most content is appropriately placed'. | 3 / 5 |
Total | 14 / 20 Passed |