CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl-labs/flask-testing

Write correct Flask tests -- app factory with test config, application context fixtures, database isolation, file uploads, auth testing, error handlers, mock.patch placement, and essential API test patterns

98

1.15x
Quality

99%

Does it follow best practices?

Impact

97%

1.15x

Average score across 5 eval scenarios

SecuritybySnyk

Passed

No known issues

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-4/

{
  "context": "Tests whether the agent proactively applies Flask testing best practices when writing tests for a recipe app with authentication. The task does NOT mention application context patterns, auth fixture patterns, or CSRF disabling -- the agent should apply these on its own.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "App context in fixture",
      "description": "The agent creates an app fixture that wraps in 'with app.app_context():' and yields from inside it.",
      "max_score": 14
    },
    {
      "name": "Auth client fixture",
      "description": "The agent creates a convenience fixture (e.g., auth_client or authenticated_client) that is already logged in, separate from the basic client fixture. Login happens through the actual /auth/login endpoint, not by faking cookies or sessions.",
      "max_score": 14
    },
    {
      "name": "Auth both paths tested",
      "description": "For protected endpoints (create, update, delete, favorite), the agent tests both the unauthenticated case (expects 401/403 or redirect) and the authenticated case (expects success).",
      "max_score": 14
    },
    {
      "name": "SQLAlchemy test isolation",
      "description": "The agent uses a test database (in-memory SQLite or separate URI) with db.create_all() in the fixture and either db.drop_all() or transaction rollback for cleanup. Each test gets a clean state.",
      "max_score": 12
    },
    {
      "name": "Test config with TESTING=True",
      "description": "The agent passes test configuration with TESTING=True, a test database URI, and SECRET_KEY. If Flask-WTF is implied, sets WTF_CSRF_ENABLED=False.",
      "max_score": 10
    },
    {
      "name": "Conftest fixture structure",
      "description": "The agent creates a conftest.py with separate app, client, and auth_client fixtures. The client depends on app.",
      "max_score": 10
    },
    {
      "name": "Authorization tests (author only)",
      "description": "The agent tests that only the recipe author can update/delete their recipe -- another authenticated user gets 403.",
      "max_score": 10
    },
    {
      "name": "Essential test patterns",
      "description": "The suite includes happy path, validation rejection, 404 for missing recipe, persistence verification, and error response consistency.",
      "max_score": 8
    },
    {
      "name": "json= for POST and get_json() for response",
      "description": "The agent uses json= for JSON requests and get_json() for parsing responses.",
      "max_score": 4
    },
    {
      "name": "Run command documented",
      "description": "The agent documents that tests are run with 'pytest -v' or 'pytest'.",
      "max_score": 4
    }
  ]
}

evals

tile.json