Fastify patterns — always apply schema-first validation, plugin encapsulation, structured error handling, hooks lifecycle, decorators, TypeScript type providers, production hardening (CORS, helmet, rate limiting), pino logging, graceful shutdown, and correct async handler patterns
89
89%
Does it follow best practices?
Impact
91%
2.75xAverage score across 5 eval scenarios
Passed
No known issues
{
"context": "Tests whether the agent proactively applies Fastify best practices when building an authentication service. The task describes what the auth system should do but says nothing about decorateRequest, preHandler hooks, TypeScript module augmentation, structured error responses, or logging patterns. The agent should apply these on its own.",
"type": "weighted_checklist",
"checklist": [
{
"name": "decorate-request-for-user",
"description": "Agent uses app.decorateRequest('user', null) to add the user property to the request object, rather than assigning arbitrary properties to the request. The agent was NOT told about decorateRequest.",
"max_score": 12
},
{
"name": "prehandler-hook-for-auth",
"description": "Agent uses a preHandler hook for JWT verification on protected routes, not onRequest (which runs before parsing). The hook is scoped correctly -- only applied to routes that need auth. The agent was NOT told which hook to use.",
"max_score": 12
},
{
"name": "typescript-module-augmentation",
"description": "Agent uses 'declare module fastify' with FastifyRequest interface augmentation to type the user property, rather than using 'as any' casts. The agent was NOT told about module augmentation.",
"max_score": 10
},
{
"name": "body-schemas-on-auth-routes",
"description": "POST /register and POST /login routes define JSON Schema for the request body with required fields, type/format constraints (e.g., format: 'email'), and additionalProperties: false. The agent was NOT told to add schema validation.",
"max_score": 12
},
{
"name": "structured-error-responses",
"description": "Auth failures (401), validation errors (400), and conflicts (409 for duplicate email) return structured error objects with code and message fields, via a custom setErrorHandler or per-route error responses. The agent was NOT told about error response format.",
"max_score": 10
},
{
"name": "auth-plugin-with-fastify-plugin",
"description": "The auth plugin that adds the decorateRequest and preHandler is wrapped with fastify-plugin (fp) so the decorator is available to all route plugins, not just the auth plugin's scope. The agent was NOT told about encapsulation breaking.",
"max_score": 10
},
{
"name": "request-log-usage",
"description": "Agent uses request.log for logging auth events (login attempts, failures, registrations) rather than console.log. Fastify is instantiated with logger: true. The agent was NOT told about logging.",
"max_score": 8
},
{
"name": "401-short-circuits-request",
"description": "When the auth hook fails (missing/invalid token), it sends a 401 response and prevents the handler from running -- either by throwing, returning reply, or using reply.send() followed by return. The agent was NOT told about hook short-circuiting.",
"max_score": 8
},
{
"name": "response-schemas",
"description": "Routes define response schemas to prevent leaking sensitive data (e.g., password hashes) in the response. The agent was NOT told about response schemas.",
"max_score": 10
},
{
"name": "graceful-shutdown",
"description": "Server startup handles SIGINT/SIGTERM by calling app.close(). The agent was NOT told about shutdown handling.",
"max_score": 8
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
skills
fastify-best-practices
verifiers