API documentation with OpenAPI/Swagger — endpoint descriptions, request/response
66
57%
Does it follow best practices?
Impact
100%
1.06xAverage score across 3 eval scenarios
Passed
No known issues
{
"context": "Tests whether the agent uses FastAPI-specific documentation patterns: summary and response_description on decorators, docstrings on handler functions, Pydantic Field(description=...) on model fields, and whether all endpoints include error responses, required/optional field distinctions, and parameter documentation.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Decorator summary param",
"description": "At least one route decorator includes a `summary=` keyword argument",
"max_score": 10
},
{
"name": "Decorator response_description",
"description": "At least one route decorator includes a `response_description=` keyword argument",
"max_score": 8
},
{
"name": "Handler docstrings",
"description": "At least two route handler functions have a docstring (triple-quoted string immediately after the `def` line)",
"max_score": 10
},
{
"name": "Pydantic Field descriptions",
"description": "At least one Pydantic model field uses `Field(description=...)` or `Field(..., description=...)`",
"max_score": 10
},
{
"name": "Error responses declared",
"description": "At least one route decorator includes `responses=` listing one or more 4xx/5xx status codes",
"max_score": 10
},
{
"name": "Required vs optional fields",
"description": "The request body Pydantic model distinguishes required fields (no default) from optional fields (using `Optional` or a default value)",
"max_score": 10
},
{
"name": "Path parameter documented",
"description": "The endpoint that retrieves a single product by ID uses a path parameter (e.g. `/products/{product_id}`)",
"max_score": 8
},
{
"name": "Query parameters present",
"description": "The list endpoint accepts at least one query parameter (e.g. category filter or page number)",
"max_score": 8
},
{
"name": "Query param typing",
"description": "Query parameters on the list endpoint have explicit Python type annotations and at least one has a default value",
"max_score": 8
},
{
"name": "All four endpoints implemented",
"description": "The application includes routes for: list products, get product by ID, create product, update stock level",
"max_score": 8
},
{
"name": "requirements.txt present",
"description": "A requirements.txt file exists and includes fastapi and an ASGI server (uvicorn or hypercorn)",
"max_score": 5
},
{
"name": "README mentions /docs",
"description": "README.md mentions `/docs` as the URL where interactive API documentation can be found",
"max_score": 5
}
]
}