Every backend service MUST use structured logging — pino/structlog/slog, JSON format, request IDs, proper log levels. console.log is never acceptable.
88
85%
Does it follow best practices?
Impact
97%
3.03xAverage score across 4 eval scenarios
Passed
No known issues
{
"context": "Tests whether the agent proactively sets up structured logging when building a payment API. The task describes only business requirements -- it never mentions logging, observability, pino, winston, or request IDs. A production-ready service should have all of these from day one.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Structured logger used (not console.log)",
"description": "Uses pino, winston, or another structured logging library for all logging. No bare console.log or console.error calls in the application code (console.log in a startup banner is acceptable only if the structured logger is also used for startup logging).",
"max_score": 15
},
{
"name": "JSON log output",
"description": "Logger is configured to produce JSON-formatted output (pino does this by default; winston needs explicit format configuration).",
"max_score": 8
},
{
"name": "Request ID middleware",
"description": "Middleware assigns a unique request ID to each incoming request -- reads from x-request-id header if present, otherwise generates a UUID or random value. The request ID is available to route handlers.",
"max_score": 12
},
{
"name": "Request ID in response header",
"description": "The assigned request ID is written back to the response via the x-request-id header.",
"max_score": 5
},
{
"name": "Request ID propagated to log entries",
"description": "Log entries inside route handlers include the request_id field -- achieved via child logger (pino), context binding, or equivalent mechanism.",
"max_score": 10
},
{
"name": "Per-request completion log",
"description": "A middleware or hook logs one structured line per completed request containing HTTP method, path, status code, and response duration in milliseconds.",
"max_score": 10
},
{
"name": "Business events logged at info level",
"description": "Key business events (payment created, payment refunded) are logged at info level with structured fields like payment_id, amount_cents, customer_id -- not string interpolation.",
"max_score": 10
},
{
"name": "404 and validation failures not logged as error",
"description": "Not-found (404) responses and validation failures (400) are logged at warn or info level -- NOT at error level. Error level is reserved for unexpected failures.",
"max_score": 8
},
{
"name": "No sensitive data logged",
"description": "Log statements do not include full request bodies, credit card details, or tokens. Payment amounts and IDs are fine, but raw body dumps are not.",
"max_score": 8
},
{
"name": "Log level configurable via environment variable",
"description": "Logger level is set from an environment variable (e.g. LOG_LEVEL) with a sensible default (info).",
"max_score": 7
},
{
"name": "Startup logged with structured logger",
"description": "Server startup (listening on port) is logged using the structured logger, not console.log.",
"max_score": 7
}
]
}