Content
82%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
A strong, expert-level catalog: every anti-pattern comes with named harness keys, an executable detection procedure, a severity, and a specific fix, and the reporting format is fully specified with a worked example. The two real critiques are that the ~435-line body could offload the worked example and more per-harness detail to the references directory, and that a few rhetorical passages and the scope table could be trimmed for token efficiency.
Suggestions
Move the worked example (promptfooconfig.yaml + eval.yml + findings table) into a references file (e.g., references/worked-example.md) and keep a two-line pointer plus the output-format spec in SKILL.md, cutting roughly 50 lines from the always-loaded body.
Consolidate the repeated per-harness detail in anti-patterns 3, 4, 5, and 8 into the existing references/harness-mapping.md, keeping only the harness-neutral detection cue inline so the checks stay one page each.
Trim the framing prose (the opening three paragraphs and the 'What this owns' table overlap with the frontmatter description) and relocate 'The judge is a model under test' to follow anti-pattern 8 so the numbered check sequence reads uninterrupted.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The body assumes Claude's competence throughout — it never explains what an eval, CI, or an assertion is, and it leads with curated specifics (exact assertion families, provider flags, thresholds) rather than tutorial prose. It is not a 5 because rhetorical framing ("can be green and still be worthless", "A threshold copied from a tutorial is a threshold nobody has validated") and the ownership/scope table partially duplicate what the description and the per-check Detect sections already establish, so a modest trim is available. It is well above the 3 anchor, where unnecessary explanation of known concepts would appear. | 4 / 5 |
Actionability | Every check gives an executable detection procedure plus a concrete fix: named Promptfoo assertion types (`equals`, `contains`, `icontains`, `regex`, `is-json` vs `llm-rubric`, `g-eval`, `factuality`, `similar` with a cosine `threshold`), copy-paste YAML for `providers`, `latency`/`cost` thresholds, the three-level grader override chain (`--grader`, `defaultTest.options.provider`, per-assertion `provider` with documented precedence), a threshold-setting procedure ("run the current known-good outputs through it... place the threshold below the worst acceptable output"), and a complete worked example with an eight-row findings table in the specified output format. This matches the fully-executable, common-cases-covered anchor; the 4 anchor's 'minor gaps' are not present. | 5 / 5 |
Workflow Clarity | The sequence is clear and checkpointed: map the five slots first (numbered 1–5), apply the eight Detect/Why/Fix checks, then emit the ordered findings table and verdict with explicit validation rules ("Every finding must point at a line", "A finding with no location is not reviewable", "A suite with an outstanding Critical finding is not passing"). It is not a 5 because there is no re-check or error-recovery loop (e.g., what to do when a slot cannot be located or when findings conflict), and "The judge is a model under test" interrupts the numbered anti-pattern sequence between 4 and 5, slightly blurring the main flow. It clears the 3 anchor easily since checkpoints are explicit rather than implicit. | 4 / 5 |
Progressive Disclosure | Structure is good: citations and the per-harness slot mapping are correctly split out to two real, one-level-deep, clearly-signaled files (references/citations.md, references/harness-mapping.md) linked both inline and in a closing References section, and headers make the 435-line body navigable. It is not a 5 because the body itself is long for a SKILL.md — the worked example and some per-harness detail in anti-patterns 3–8 are candidates for reference files — so the split is good but not maximally considerate of the context window. It is well above the 3 anchor, where bulk reference material would sit inline with weak signaling. | 4 / 5 |
Total | 17 / 20 Passed |