Fastify patterns — always apply schema-first validation, plugin encapsulation, structured error handling, hooks lifecycle, decorators, TypeScript type providers, production hardening (CORS, helmet, rate limiting), pino logging, graceful shutdown, and correct async handler patterns
89
89%
Does it follow best practices?
Impact
91%
2.75xAverage score across 5 eval scenarios
Passed
No known issues
{
"context": "Tests whether the agent proactively applies Fastify best practices across multiple concerns when building a file metadata service with a webhook endpoint. The task says nothing about schema validation, idempotency, error handling patterns, security plugins, or webhook-specific patterns. The agent should apply these on its own.",
"type": "weighted_checklist",
"checklist": [
{
"name": "strict-body-schemas",
"description": "POST routes (file creation and webhook) define JSON Schema for request bodies with required fields, type constraints (format: 'uuid', enum for status/mimeType), and additionalProperties: false. The agent was NOT told to add validation.",
"max_score": 12
},
{
"name": "webhook-schema-strict",
"description": "The webhook endpoint has a strict body schema that validates the incoming payload structure (fileId, status with enum, optional error) and rejects unexpected fields. The agent was NOT told to validate webhooks strictly.",
"max_score": 10
},
{
"name": "response-schemas",
"description": "Routes define response schemas for success status codes to enable fast serialization and prevent data leaks. The agent was NOT told about response schemas.",
"max_score": 8
},
{
"name": "separate-plugins-for-files-and-webhooks",
"description": "File routes and webhook routes are in separate Fastify plugins registered with app.register(), providing encapsulation. The agent was NOT told about plugin separation.",
"max_score": 8
},
{
"name": "custom-error-handler",
"description": "A custom error handler via setErrorHandler returns structured errors with code/message, handling validation errors distinctly from 404s and server errors. The agent was NOT told about error handling.",
"max_score": 10
},
{
"name": "not-found-on-missing-resources",
"description": "GET/PATCH/DELETE by ID routes throw proper 404 errors when the file record is not found. The agent was NOT told how to handle missing resources.",
"max_score": 8
},
{
"name": "webhook-idempotency",
"description": "The webhook handler considers idempotency -- checking if the file status was already updated or using the fileId to prevent duplicate processing. The agent was NOT told about idempotency.",
"max_score": 10
},
{
"name": "webhook-event-logging",
"description": "Webhook events are logged with structured context (fileId, new status, source) using request.log, not console.log. The agent was NOT told about webhook logging.",
"max_score": 8
},
{
"name": "pino-logger-enabled",
"description": "Fastify is instantiated with logger enabled. All logging uses request.log or app.log. The agent was NOT told about logging.",
"max_score": 6
},
{
"name": "security-plugins",
"description": "Agent registers at least one security plugin (@fastify/cors, @fastify/helmet, or @fastify/rate-limit). The agent was NOT told about security hardening.",
"max_score": 8
},
{
"name": "graceful-shutdown",
"description": "Server handles SIGINT/SIGTERM signals by calling app.close(). The agent was NOT told about shutdown handling.",
"max_score": 6
},
{
"name": "querystring-schemas",
"description": "The GET /api/files list route defines a querystring schema for filter parameters (uploadedBy, status, mimeType) with proper types. The agent was NOT told to validate query params.",
"max_score": 6
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
skills
fastify-best-practices
verifiers