Content
88%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
A high-quality operational runbook: dense with verified, non-inferable specifics, fully executable commands, and a clearly sequenced tiered workflow with explicit validation checkpoints and error recovery. The only weaknesses are minor repetition and the near-total inlining of content that could partially live in reference files.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | Nearly every passage carries non-obvious, verified, org-specific detail (resource GUIDs, api-version gotchas, log-size heuristics) with no padding of concepts Claude already knows. Minor duplication could be trimmed: the green-build/retried-attempt warning appears both as a taxonomy callout ("Green builds hide failures too") and again in Tier 3 ("⚠️ Green builds can still hide failures"), and the bulk-log-cost point is made twice ("treat it as a last resort" intro + "just know it's the costly path"). | 4 / 5 |
Actionability | Fully executable throughout: exact REST URLs with required api-versions ("vstmr.dev.azure.com/...resultsbybuild?...api-version=7.1-preview.1"), concrete MCP invocations with parameters ("ado-pipelines_build action=get_status buildId={ID} project=internal"), runnable PowerShell token/deep-link snippets, a real helper script ("scripts/Parse-AdoTestLog.ps1 -Path <temp-file> [-Context 30]"), and a ready-to-paste bug Description template. Not 4 because the commands cover the common cases end-to-end with no gaps requiring invention. | 5 / 5 |
Workflow Clarity | The tiered escalation is explicitly gated ("Descend a tier only when the one above can't answer the question"), each tier has numbered steps, and validation checkpoints are pervasive: snapshot-before-click, match logs "on the first line / the test:realsvc:<driver>:report command rather than trusting size", deterministic-vs-flaky verdict criteria, "Always check for retries before declaring a build clean", and "Confirm scope with the user first" before the write operation (bug filing). Error-recovery loops are also present for auth failures (tenant mismatch → az login, az.cmd not on PATH → full path). | 5 / 5 |
Progressive Disclosure | Good structure: clear section headers, a navigable tier table, and the one bundle file that exists (scripts/Parse-AdoTestLog.ps1, verified present) is clearly referenced with its invocation. Not 5 because virtually all ~300 lines live inline in SKILL.md — the "Reference specifics — Real Service End to End Tests" section and the bug template are natural candidates for references/ files; not 3 because what exists is well-organized and the single reference is clearly signaled. | 4 / 5 |
Total | 18 / 20 Passed |