CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl-labs/flask-testing

Write correct Flask tests -- app factory with test config, application context fixtures, database isolation, file uploads, auth testing, error handlers, mock.patch placement, and essential API test patterns

98

1.15x
Quality

99%

Does it follow best practices?

Impact

97%

1.15x

Average score across 5 eval scenarios

SecuritybySnyk

Passed

No known issues

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-3/

{
  "context": "Tests whether the agent proactively applies Flask file upload testing patterns. The task does NOT mention BytesIO, multipart/form-data details, or the data= vs json= distinction -- the agent should apply these on its own.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "BytesIO file upload pattern",
      "description": "The agent uses (BytesIO(bytes), filename) tuples in the data dict for file uploads, not raw strings, file paths, or json=.",
      "max_score": 16
    },
    {
      "name": "data= with content_type multipart",
      "description": "File upload tests use data= parameter with content_type='multipart/form-data', not json= for the upload endpoint.",
      "max_score": 14
    },
    {
      "name": "Valid and invalid file type tests",
      "description": "The agent tests uploading allowed file types (PDF, PNG, JPEG, CSV) AND tests that disallowed types (e.g., .exe, .sh) are rejected with 400.",
      "max_score": 14
    },
    {
      "name": "App context in fixture",
      "description": "The agent creates an app fixture wrapped in 'with app.app_context():' and yields from inside it.",
      "max_score": 12
    },
    {
      "name": "Test config with TESTING=True",
      "description": "The agent passes test config with TESTING=True, a test database, and a temporary UPLOAD_FOLDER (using tempfile or tmp_path).",
      "max_score": 10
    },
    {
      "name": "Auth via API key header",
      "description": "The agent authenticates test requests using the Authorization header with the API key, as specified. Creates an auth fixture or helper.",
      "max_score": 10
    },
    {
      "name": "Auth both paths tested",
      "description": "Tests verify that unauthenticated requests (missing or invalid API key) are rejected, and authenticated requests succeed.",
      "max_score": 8
    },
    {
      "name": "Owner-only access tested",
      "description": "Tests verify that one user cannot access or delete another user's documents.",
      "max_score": 8
    },
    {
      "name": "Conftest fixture structure",
      "description": "Separate app, client fixtures in conftest.py with proper dependency chain.",
      "max_score": 4
    },
    {
      "name": "Database isolation per test",
      "description": "Function-scoped fixtures ensure each test gets a clean database. No scope='session' on database fixtures.",
      "max_score": 4
    }
  ]
}

evals

tile.json