Content
85%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
An exceptionally actionable and well-sequenced skill body: every phase of authoring, validating, and running a metric spec carries executable commands, complete examples, and hard-won operational gotchas, with almost no filler. Its one structural weakness is that it is a single long file — the assertion API, date-window design, and CI details are reference material that would be better split into one-level-deep bundle files.
Suggestions
Move the assertion-helper API (r.row/equals/contains/nonempty/check, one/some selectors, approx, the per-view required-field list) into a references/assertions.md and keep only 2–3 representative examples inline in SKILL.md.
Split the 'Date-window test design' BVA/equivalence-partition guidance and the 'New bronze table for a not-yet-seeded connector' procedure into references/ files, linked from a short 'when a metric is date-windowed' / 'when the connector isn't seeded' pointer in the body.
Trim knowledge Claude already has, e.g. 'A record is a field map' and the textbook definition of an even-sample median, keeping only the repo-specific rule (quantileExactInclusive vs quantilesExact) that the test must compute with.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The body is dense with genuinely non-obvious, repo-specific knowledge (isolation ledger semantics, cohort resolution via department string, MIN_PEER_N rule, shared-venv conflict) and almost never explains things Claude already knows. A few statements could be trimmed, e.g. "A record is a field map" and the textbook aside "A `median` is the textbook one … an even sample averages its two middles", which puts it at the efficient-with-minor-trimmables anchor rather than the every-token-earns-its-place anchor. | 4 / 5 |
Actionability | Guidance is fully executable: a complete worked YAML fixture (templates + bronze with $ref, overrides and a duplicate row), a complete copy-paste-ready pytest module, exact commands (uv run --project tests --frozen --group datapath …, ./dev-compose.sh test-stand … with flags), and runnable validation snippets including the schema-coverage diff loop. The common cases (scaffold a new test, validate without a stand, run one shard) are each covered by a specific command. | 5 / 5 |
Workflow Clarity | The workflow is clearly sequenced as phases — "Scaffolding a new test" (numbered steps 1–6), "Validating a test (no stand needed)" with runnable checks, "Running" with per-command semantics, and the coverage gate — and the Gotchas section supplies explicit error-recovery guidance (SeededWarehouseError → raise a minimal instance; mid-run edit → let the run finish, then re-run; ruff only outside a running suite). This matches the anchor with explicit validation steps, feedback loops, and checklists; the level-4 anchor's "minor validation gaps" understates how thorough the validation section is. | 5 / 5 |
Progressive Disclosure | No bundle files exist (references/, scripts/, assets/ are absent), and the SKILL.md is a ~460-line monolith that inlines substantial reference material — the full assertion-helper API (~40 lines), the date-window BVA/equivalence design, the CI shard topology, and the new-bronze-table procedure — that the rubric rationale says belongs one level deep. Structure and signaling are good (clear headers, an explicit "Source of truth (reference — open only if you need the detail)" pointer to the library), which keeps it above the level-2 anchor, but the bulk of inline detail places it at "some structure; content that should be separate is inline" rather than level 4's "most content is appropriately placed". | 3 / 5 |
Total | 17 / 20 Passed |