Content
50%Weight 40%Scale 1-5Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
The body is well-structured and gives solid coverage of eval types, grader types, metrics (pass@k, pass^k), a define→implement→evaluate→report workflow, and best practices, with several executable code-grader examples. Its weaknesses are placeholder pseudocode in the evaluate step, an undefined /eval command surface with no backing scripts, no failure-recovery feedback loop, and repeated template/example content that inflates length without adding new information.
Suggestions
Replace the placeholder evaluate step ("[Run each capability eval, record PASS/FAIL]") with executable commands, and define what /eval define/check/report actually invoke since no scripts bundle them (actionability).
Add an explicit feedback loop to the workflow, e.g. "If any regression eval fails, fix and re-run until pass^3 = 100% before generating the report" (workflow_clarity).
Deduplicate the eval format, which appears three times (评估类型, 工作流程 定义, and the 示例 section): keep one template plus one worked example and move extras to a references/ file (conciseness, progressive_disclosure).
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The body is mostly tight lists and templates with no basic-concept padding, but the eval format is demonstrated three times (the "[能力评估：功能名称]" template, the "评估定义：功能-xyz" workflow template, and the full "示例：添加身份验证" section) and the 理念 section explains methodology philosophy. This is "mostly efficient but could be tightened" rather than the "minor trim" of anchor 4. | 3 / 5 |
Actionability | It provides genuinely executable grader commands ("grep -q \"export function handleAuth\" src/auth.ts && echo \"PASS\"", "npm test -- --testPathPattern=\"auth\"") and copy-paste templates, but the core evaluate step is placeholder pseudocode ("[Run each capability eval, record PASS/FAIL]" and a bare "# Generate report"), the "/eval define|check|report" commands have no defined implementation anywhere in the bundle, and a template contains the typo "标准 标准 3". This lands on "some concrete guidance but incomplete". | 3 / 5 |
Workflow Clarity | The four-phase sequence (定义 → 实现 → 评估 → 报告) is clearly ordered with per-phase templates, but the evaluate step runs a batch of eval suites with no explicit validation feedback loop — nothing says "if an eval fails, fix and re-run before reporting". Per the rubric's batch-operation cap ("missing validation/verification steps in workflows involving batch operations should cap workflow clarity at 3"), this cannot exceed 3 despite the clear sequence. | 3 / 5 |
Progressive Disclosure | No bundle files exist and the single file is well-sectioned with clear headers, but at ~270 lines the repeated eval templates and the full worked "示例：添加身份验证" example are inlined content that would be better split into a references/ file (e.g. EXAMPLES.md or TEMPLATES.md). This matches "some structure but could be better organized"; not 4 because content that should be separate is inline. | 3 / 5 |
Total | 12 / 20 Passed |