Creates, structures, and reviews technical documentation following the Diátaxis framework (tutorials, how-to guides, reference, and explanation pages). Use when a user needs to write or reorganize docs, structure a tutorial vs. a how-to guide, build reference docs or API documentation, create explanation pages, choose between Diátaxis documentation types, or improve existing documentation structure. Trigger terms include: documentation structure, Diátaxis, tutorials vs how-to guides, organize docs, user guide, reference docs, technical writing.
91
87%
Does it follow best practices?
Impact
96%
1.11xAverage score across 5 eval scenarios
Passed
No known issues
{
"context": "Tests whether the agent follows reference documentation Diátaxis patterns: noun-based title naming the subject, consistent per-entry format (name → type → default → description → example), stating facts without instructional prose, and making individual entries independently scannable.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Noun-based title",
"description": "The document title names the subject directly (e.g., 'Configuration Options', 'deploykit.yml Reference') rather than framing it as a task or concept",
"max_score": 8
},
{
"name": "Name field per entry",
"description": "Every configuration entry includes the exact option name (e.g., as a heading, bold text, or code span)",
"max_score": 8
},
{
"name": "Type field per entry",
"description": "Every configuration entry includes the data type (e.g., string, integer, boolean, list)",
"max_score": 10
},
{
"name": "Default value per entry",
"description": "Every configuration entry includes the default value (or explicitly states 'required' / 'no default')",
"max_score": 10
},
{
"name": "Description per entry",
"description": "Every configuration entry includes a brief factual description of what the option controls",
"max_score": 8
},
{
"name": "Example per entry",
"description": "At least 8 out of the documented entries include a usage example (e.g., a YAML snippet showing the key with a sample value)",
"max_score": 10
},
{
"name": "Consistent format",
"description": "All entries follow the same structural format — the presentation of name, type, default, description, and example is consistent across entries (not ad-hoc prose for some and structured for others)",
"max_score": 12
},
{
"name": "Facts not instructions",
"description": "The entry descriptions state what the option does rather than instructing the reader to do something (e.g., 'Controls the maximum number of retries' not 'Set this to 3 to get three retries')",
"max_score": 10
},
{
"name": "No step-by-step mixing",
"description": "The document does NOT include a numbered walkthrough or tutorial section on how to write a deploykit.yml file from scratch",
"max_score": 8
},
{
"name": "No conceptual mixing",
"description": "The document does NOT include a section explaining why deploykit uses a YAML config or the philosophy behind the configuration design",
"max_score": 8
},
{
"name": "Entries independently scannable",
"description": "Each entry is self-contained and could be read in isolation — it does not require reading adjacent entries to understand its meaning",
"max_score": 8
}
]
}