Write correct Flask tests -- app factory with test config, application context fixtures, database isolation, file uploads, auth testing, error handlers, mock.patch placement, and essential API test patterns
98
99%
Does it follow best practices?
Impact
97%
1.15xAverage score across 5 eval scenarios
Passed
No known issues
{
"context": "Tests whether the agent proactively applies Flask file upload testing patterns. The task does NOT mention BytesIO, multipart/form-data details, or the data= vs json= distinction -- the agent should apply these on its own.",
"type": "weighted_checklist",
"checklist": [
{
"name": "BytesIO file upload pattern",
"description": "The agent uses (BytesIO(bytes), filename) tuples in the data dict for file uploads, not raw strings, file paths, or json=.",
"max_score": 16
},
{
"name": "data= with content_type multipart",
"description": "File upload tests use data= parameter with content_type='multipart/form-data', not json= for the upload endpoint.",
"max_score": 14
},
{
"name": "Valid and invalid file type tests",
"description": "The agent tests uploading allowed file types (PDF, PNG, JPEG, CSV) AND tests that disallowed types (e.g., .exe, .sh) are rejected with 400.",
"max_score": 14
},
{
"name": "App context in fixture",
"description": "The agent creates an app fixture wrapped in 'with app.app_context():' and yields from inside it.",
"max_score": 12
},
{
"name": "Test config with TESTING=True",
"description": "The agent passes test config with TESTING=True, a test database, and a temporary UPLOAD_FOLDER (using tempfile or tmp_path).",
"max_score": 10
},
{
"name": "Auth via API key header",
"description": "The agent authenticates test requests using the Authorization header with the API key, as specified. Creates an auth fixture or helper.",
"max_score": 10
},
{
"name": "Auth both paths tested",
"description": "Tests verify that unauthenticated requests (missing or invalid API key) are rejected, and authenticated requests succeed.",
"max_score": 8
},
{
"name": "Owner-only access tested",
"description": "Tests verify that one user cannot access or delete another user's documents.",
"max_score": 8
},
{
"name": "Conftest fixture structure",
"description": "Separate app, client fixtures in conftest.py with proper dependency chain.",
"max_score": 4
},
{
"name": "Database isolation per test",
"description": "Function-scoped fixtures ensure each test gets a clean database. No scope='session' on database fixtures.",
"max_score": 4
}
]
}