CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl-labs/pytest-api-testing

Pytest patterns for Python APIs -- httpx AsyncClient, conftest fixtures, database isolation, parametrize edge cases, error response testing, auth flows, factory fixtures

99

1.23x
Quality

99%

Does it follow best practices?

Impact

100%

1.23x

Average score across 5 eval scenarios

SecuritybySnyk

Passed

No known issues

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-2/

{
  "context": "Tests whether the agent correctly structures database fixtures with proper isolation: session-scoped engine, function-scoped session, cleanup before each test (not after), autouse=True on the cleanup fixture, and FastAPI dependency override for injecting the test session.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Session-scoped engine",
      "description": "The db_engine (or equivalent) fixture uses scope=\"session\" since schema creation is expensive and read-only",
      "max_score": 10
    },
    {
      "name": "Function-scoped session",
      "description": "The db_session (or equivalent per-test database session) fixture does NOT use scope=\"session\" or scope=\"module\"",
      "max_score": 10
    },
    {
      "name": "Cleanup before yield",
      "description": "The database cleanup fixture deletes/truncates rows BEFORE the yield (not only after), so a mid-test failure still leaves a clean state for the next test",
      "max_score": 15
    },
    {
      "name": "autouse cleanup",
      "description": "The database cleanup fixture uses autouse=True so every test automatically gets a fresh state",
      "max_score": 15
    },
    {
      "name": "Dependency override",
      "description": "Tests inject the test db session using app.dependency_overrides[get_db] (not by patching the module or mocking the database)",
      "max_score": 15
    },
    {
      "name": "No database mock",
      "description": "The tests do NOT mock or stub the database layer — they use a real SQLite in-memory or file-based engine",
      "max_score": 10
    },
    {
      "name": "Override cleared after",
      "description": "app.dependency_overrides is cleared (e.g., .clear()) after the client fixture teardown to avoid leaking overrides",
      "max_score": 10
    },
    {
      "name": "in-memory or transaction isolation",
      "description": "Tests use SQLite :memory: or transaction rollback to avoid persisting data to disk between test runs",
      "max_score": 10
    },
    {
      "name": "Fixtures in conftest",
      "description": "All database and client fixtures are defined in tests/conftest.py (not inside the test file itself)",
      "max_score": 5
    }
  ]
}

evals

tile.json