CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl-labs/pytest-api-testing

Pytest patterns for Python APIs -- httpx AsyncClient, conftest fixtures, database isolation, parametrize edge cases, error response testing, auth flows, factory fixtures

99

1.23x
Quality

99%

Does it follow best practices?

Impact

100%

1.23x

Average score across 5 eval scenarios

SecuritybySnyk

Passed

No known issues

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-1/

{
  "context": "Tests whether the agent correctly sets up a FastAPI test suite using httpx.AsyncClient with ASGITransport, configures pytest for async operation, places fixtures in conftest.py, and avoids using requests or urllib as the HTTP client.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Uses AsyncClient",
      "description": "The conftest.py client fixture uses httpx.AsyncClient (not requests.Session or urllib)",
      "max_score": 15
    },
    {
      "name": "Uses ASGITransport",
      "description": "The client fixture passes ASGITransport(app=app) to AsyncClient as the transport parameter",
      "max_score": 15
    },
    {
      "name": "asyncio_mode auto",
      "description": "pyproject.toml includes asyncio_mode = \"auto\" under [tool.pytest.ini_options]",
      "max_score": 15
    },
    {
      "name": "pytest-asyncio installed",
      "description": "pytest-asyncio is listed as a test dependency (requirements-test.txt or pyproject.toml)",
      "max_score": 10
    },
    {
      "name": "httpx installed",
      "description": "httpx is listed as a test dependency",
      "max_score": 10
    },
    {
      "name": "Fixture in conftest.py",
      "description": "The client fixture is defined in tests/conftest.py (not inside a test file or imported manually)",
      "max_score": 10
    },
    {
      "name": "Function-scoped client",
      "description": "The client fixture does NOT use scope=\"session\" or scope=\"module\" — it defaults to function scope",
      "max_score": 10
    },
    {
      "name": "base_url set",
      "description": "AsyncClient is initialized with base_url=\"http://test\" (or equivalent non-production URL)",
      "max_score": 10
    },
    {
      "name": "No requests/urllib import",
      "description": "No test file imports requests or urllib for making HTTP calls",
      "max_score": 5
    }
  ]
}

evals

scenario-1

criteria.json

task.md

tile.json