Fastify patterns — always apply schema-first validation, plugin encapsulation, structured error handling, hooks lifecycle, decorators, TypeScript type providers, production hardening (CORS, helmet, rate limiting), pino logging, graceful shutdown, and correct async handler patterns
89
89%
Does it follow best practices?
Impact
91%
2.75xAverage score across 5 eval scenarios
Passed
No known issues
{
"context": "Tests whether the agent proactively applies Fastify TypeScript and schema patterns when building a blog API. The task mentions TypeScript type provider in passing (file list) but says nothing about as const, module augmentation, schema reuse with $ref, response schemas, or async patterns. The agent should apply these on its own.",
"type": "weighted_checklist",
"checklist": [
{
"name": "type-provider-configured",
"description": "Agent configures a Fastify type provider (JsonSchemaToTsProvider or TypeBoxTypeProvider) using withTypeProvider() so schema types are automatically inferred. The agent was NOT told how to configure the type provider.",
"max_score": 10
},
{
"name": "as-const-on-schemas",
"description": "JSON Schema objects use 'as const' assertion so TypeScript infers literal types from the schema. The agent was NOT told about as const.",
"max_score": 8
},
{
"name": "shared-schemas-with-ref",
"description": "Agent uses app.addSchema() to define shared schema components (post, comment) and references them with $ref in route schemas, rather than duplicating schema definitions. The agent was NOT told about schema reuse.",
"max_score": 10
},
{
"name": "comprehensive-body-schemas",
"description": "All POST/PUT routes define body schemas with required fields, minLength/maxLength constraints matching the spec, enum for tags if applicable, and additionalProperties: false. The agent was NOT told to add validation beyond the field descriptions.",
"max_score": 12
},
{
"name": "querystring-schemas-with-pagination",
"description": "GET list routes define querystring schemas for pagination (page, limit with maximum: 50) and filter parameters (published, tag) with proper types. The agent was NOT told to validate query params.",
"max_score": 8
},
{
"name": "response-schemas-prevent-leaks",
"description": "Routes define response schemas for success status codes. This prevents accidental field leaks and enables fast serialization. The agent was NOT told about response schemas.",
"max_score": 10
},
{
"name": "no-as-any-casting",
"description": "Agent does not use 'as any' or manual type assertions on request.body, request.params, or request.query -- types flow from schemas via the type provider. The agent was NOT told to avoid casting.",
"max_score": 8
},
{
"name": "plugins-with-correct-typing",
"description": "Plugin functions use FastifyPluginAsync type from the fastify package for their signature. The agent was NOT told about plugin typing.",
"max_score": 6
},
{
"name": "custom-error-handler",
"description": "A custom error handler with structured responses is set up, with validation errors handled separately. The agent was NOT told about error handling.",
"max_score": 8
},
{
"name": "not-found-errors",
"description": "GET/PUT/DELETE by ID routes throw a proper 404 error when the resource is not found, using @fastify/error or a custom error class with statusCode. The agent was NOT told how to handle missing resources.",
"max_score": 8
},
{
"name": "pino-logging",
"description": "Fastify is created with logger enabled. Code uses request.log and app.log, never console.log. The agent was NOT told about logging.",
"max_score": 6
},
{
"name": "graceful-shutdown",
"description": "Server startup handles SIGINT/SIGTERM by calling app.close(). The agent was NOT told about shutdown.",
"max_score": 6
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
skills
fastify-best-practices
verifiers