Content
78%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
The body is an efficient, well-organized set of evaluation discipline guidance with genuine non-obvious insights and a clear phase structure. Its main gap is actionability: it tells Claude how to think about cases and graders but never shows a concrete promptfoo case or assertion example, leaving the authoring step to be improvised.
Suggestions
Add a minimal worked example in 'Build a useful case and grader' — a short promptfooconfig.yaml snippet with one test case and a deterministic assertion (e.g., an assert-type with a threshold) — so the core authoring task is concrete rather than principle-only.
Turn 'Define what is being evaluated' through 'Run and interpret' into an explicitly numbered sequence, and add a short error-recovery loop (what to do on provider failure, flaky cases, or failing grader checks) to strengthen the workflow's feedback checkpoints.
Name the common promptfoo assertion types or link where to find them (e.g., 'assertions: contains, javascript, llm-rubric') so grader guidance maps directly to real configuration syntax.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The body is lean with no padding and no explanation of concepts Claude already knows — every line is non-obvious judgment guidance ('A client passing a case does not prove the harness caused the behavior', 'Unavailable providers are not passes'). It assumes Claude's competence and each directive earns its tokens, matching the top anchor rather than the 4 anchor's 'minor instances of over-explanation'. | 5 / 5 |
Actionability | There is some concrete guidance — 'inspect evals/promptfoo/promptfooconfig.yaml', 'make install-promptfoo', 'make eval-promptfoo', 'pin the evaluator and commit the dependency lockfile' — but it is incomplete for the core task: no example promptfoo case, assertion syntax, or config snippet showing how to actually author an evaluation. It is not a 4 because the central 'build a case and grader' section stays at the level of principles ('Prefer deterministic assertions for structure, exact facts, and prohibited actions') without a concrete worked example, and not a 2 because commands and file paths do give real, executable hooks. | 3 / 5 |
Workflow Clarity | A clear define → build → run-and-interpret sequence with most checkpoints present: 'Exercise the grader with a known correct response and plausible incorrect responses before trusting it', 'check its judgments against examples reviewed by a person', and 'Report behavioral failures separately from provider errors, timeouts, missing credentials'. It falls short of the 5 anchor because error-recovery feedback loops (what to do when a run fails or flakes) are implied rather than explicit, and the steps are prose sections rather than an explicit ordered sequence. | 4 / 5 |
Progressive Disclosure | No bundle files exist (references/, scripts/, assets/ are absent), and the skill needs none: the ~68-line body is self-contained, cleanly sectioned, and everything it contains is core guidance that belongs in SKILL.md. The one file pointer it gives (the Arnesto config path and make targets) is clearly signaled and one level deep, matching the well-organized-sections standard for simple self-contained skills. | 5 / 5 |
Total | 17 / 20 Passed |