Write correct Flask tests -- app factory with test config, application context fixtures, database isolation, file uploads, auth testing, error handlers, mock.patch placement, and essential API test patterns
98
99%
Does it follow best practices?
Impact
97%
1.15xAverage score across 5 eval scenarios
Passed
No known issues
{
"context": "Tests whether the agent proactively applies Flask testing best practices when writing tests for a recipe app with authentication. The task does NOT mention application context patterns, auth fixture patterns, or CSRF disabling -- the agent should apply these on its own.",
"type": "weighted_checklist",
"checklist": [
{
"name": "App context in fixture",
"description": "The agent creates an app fixture that wraps in 'with app.app_context():' and yields from inside it.",
"max_score": 14
},
{
"name": "Auth client fixture",
"description": "The agent creates a convenience fixture (e.g., auth_client or authenticated_client) that is already logged in, separate from the basic client fixture. Login happens through the actual /auth/login endpoint, not by faking cookies or sessions.",
"max_score": 14
},
{
"name": "Auth both paths tested",
"description": "For protected endpoints (create, update, delete, favorite), the agent tests both the unauthenticated case (expects 401/403 or redirect) and the authenticated case (expects success).",
"max_score": 14
},
{
"name": "SQLAlchemy test isolation",
"description": "The agent uses a test database (in-memory SQLite or separate URI) with db.create_all() in the fixture and either db.drop_all() or transaction rollback for cleanup. Each test gets a clean state.",
"max_score": 12
},
{
"name": "Test config with TESTING=True",
"description": "The agent passes test configuration with TESTING=True, a test database URI, and SECRET_KEY. If Flask-WTF is implied, sets WTF_CSRF_ENABLED=False.",
"max_score": 10
},
{
"name": "Conftest fixture structure",
"description": "The agent creates a conftest.py with separate app, client, and auth_client fixtures. The client depends on app.",
"max_score": 10
},
{
"name": "Authorization tests (author only)",
"description": "The agent tests that only the recipe author can update/delete their recipe -- another authenticated user gets 403.",
"max_score": 10
},
{
"name": "Essential test patterns",
"description": "The suite includes happy path, validation rejection, 404 for missing recipe, persistence verification, and error response consistency.",
"max_score": 8
},
{
"name": "json= for POST and get_json() for response",
"description": "The agent uses json= for JSON requests and get_json() for parsing responses.",
"max_score": 4
},
{
"name": "Run command documented",
"description": "The agent documents that tests are run with 'pytest -v' or 'pytest'.",
"max_score": 4
}
]
}