Content
50%Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
The body is well-structured with specific metrics, thresholds, and a useful error-handling table, but it stays at a procedural-descriptive level: no executable code, no inline validation checkpoints, and bundled resources are not navigated to.
Suggestions
Add copy-paste-ready code or command examples showing how to run the bundled scripts (e.g. `python scripts/validate_model.py --model model.pkl --data data.csv --attr race,gender`) so the guidance is fully executable.
Embed validation checkpoints in the workflow (e.g. after step 4, 'verify each group has >= 30 samples before classifying severity; if not, apply the Insufficient group sample size handling') rather than relegating all checks to a separate table.
Link the bundled files from the body — point to scripts/generate_report.py and assets/report_template.md in the Output section, and reference validate_dataset.py in step 1 — so progressive disclosure is actually wired up.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The body is mostly efficient and assumes Claude's competence (it names metrics like demographic parity and TPR/FPR without explaining them), but the Overview restates the description and the three Examples are narrative-heavy, so it could be tightened. | 2 / 3 |
Actionability | It gives concrete specifics (four-fifths 0.80 threshold, severity bands, proxy r > 0.3, named library calls like ExponentiatedGradient) but contains no executable code blocks and never shows how to invoke the bundled scripts, leaving guidance incomplete. | 2 / 3 |
Workflow Clarity | A clear 10-step sequence is present and an Error Handling table supplies recovery guidance, but validation checkpoints are not embedded inline in the workflow ('validate then only proceed'), which the rubric requires for a score of 3 on batch/analysis operations. | 2 / 3 |
Progressive Disclosure | Sections are well organized, but the bundled scripts (validate_model.py, validate_dataset.py, generate_report.py) and assets (report_template.md, example files) are never referenced or linked from the body, leaving those reference files orphaned and poorly signaled. | 2 / 3 |
Total | 8 / 12 Passed |