Production config management for any backend — centralized config module, env vars, no hardcoded secrets, fail-fast validation
86
77%
Does it follow best practices?
Impact
100%
1.88xAverage score across 5 eval scenarios
Passed
No known issues
{
"context": "Tests whether the agent proactively implements proper configuration management when building a Flask weather API that connects to OpenWeatherMap, Redis, and SQLite. The task does not mention config management, environment variables, or .env files.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Centralized config module",
"description": "A dedicated config module exists (e.g. app/config.py or config.py) that consolidates all configuration. Application files import config from this module rather than calling os.getenv() directly in route handlers, weather_client.py, cache.py, or db.py.",
"max_score": 18
},
{
"name": "OpenWeatherMap API key from environment",
"description": "The OpenWeatherMap API key comes from an environment variable (e.g. OPENWEATHERMAP_API_KEY or OWM_API_KEY), not hardcoded. The weather client uses the env-based config value.",
"max_score": 16
},
{
"name": "Required config fails fast",
"description": "Required config values (OpenWeatherMap API key) are validated at startup -- if missing, the app raises an error immediately rather than failing when the first weather request is made.",
"max_score": 14
},
{
"name": "Redis URL from environment",
"description": "The Redis connection URL/host comes from an environment variable with a sensible default (e.g. localhost:6379 for development), not hardcoded as a fixed production URL.",
"max_score": 12
},
{
"name": "No hardcoded secrets",
"description": "No API keys or credentials are hardcoded in source files. All sensitive values come from environment variables through the config module.",
"max_score": 12
},
{
"name": "Env file gitignored",
"description": ".env is listed in .gitignore so secrets are not committed to the repository",
"max_score": 10
},
{
"name": ".env.example exists",
"description": "A .env.example file documents required environment variables (OPENWEATHERMAP_API_KEY, REDIS_URL, etc.) without real secret values",
"max_score": 8
},
{
"name": "Weather endpoints functional",
"description": "The weather and favorites endpoints are implemented with caching, rate limiting, and the specified business rules",
"max_score": 10
}
]
}