Write correct Flask tests -- app factory with test config, application context fixtures, database isolation, file uploads, auth testing, error handlers, mock.patch placement, and essential API test patterns
98
99%
Does it follow best practices?
Impact
97%
1.15xAverage score across 5 eval scenarios
Passed
No known issues
{
"context": "Tests whether the agent proactively applies Flask testing best practices when writing a blog API test suite. The task does NOT mention application context, fixture structure, or database isolation patterns -- the agent should apply these on its own.",
"type": "weighted_checklist",
"checklist": [
{
"name": "App context in fixture",
"description": "The agent creates an app fixture that wraps in 'with app.app_context():' and yields from inside it, ensuring the test client operates within an application context.",
"max_score": 16
},
{
"name": "Conftest fixture structure",
"description": "The agent creates a conftest.py with separate app, client fixtures. The client fixture depends on the app fixture. The app fixture handles database initialization.",
"max_score": 14
},
{
"name": "Test config with TESTING=True",
"description": "The agent passes test configuration to create_app with TESTING=True and a separate/in-memory database, not using the production database.",
"max_score": 12
},
{
"name": "Database isolation per test",
"description": "The agent uses function-scoped fixtures (default scope) so each test gets a fresh database. Does NOT use scope='session' or scope='module' for the app/database fixture.",
"max_score": 12
},
{
"name": "Auth tests both paths",
"description": "Tests verify both authenticated and unauthenticated access -- e.g., unauthenticated POST to /api/posts returns 401/403, authenticated POST succeeds. Auth is done through the login endpoint, not by faking cookies.",
"max_score": 12
},
{
"name": "Essential test patterns",
"description": "The test suite includes: happy path (200 for GET list), validation rejection (400 for empty/invalid POST), 404 for missing posts, and persistence verification (create then retrieve).",
"max_score": 10
},
{
"name": "Error response body checked",
"description": "Tests check error response JSON body structure (not just status codes), verifying that error responses contain an error key with a message.",
"max_score": 8
},
{
"name": "json= for POST requests",
"description": "The agent uses json= parameter for JSON POST/PUT requests to the test client, not data= with manual content type headers.",
"max_score": 6
},
{
"name": "get_json() for responses",
"description": "The agent uses res.get_json() to parse Flask test client responses, not res.json or json.loads(res.data).",
"max_score": 6
},
{
"name": "Run command documented",
"description": "The agent documents that tests are run with 'pytest -v' or 'pytest'.",
"max_score": 4
}
]
}