CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl-labs/django-testing

Write correct Django tests — TestCase vs TransactionTestCase, setUpTestData, factory-boy, assertNumQueries, mock.patch placement, and DRF APITestCase patterns

99

1.33x
Quality

99%

Does it follow best practices?

Impact

99%

1.33x

Average score across 2 eval scenarios

SecuritybySnyk

Passed

No known issues

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-2/

{
  "context": "Tests whether the agent proactively applies Django testing best practices for a complex checkout flow with external services, on_commit hooks, and settings-dependent behavior. The task describes the system but does NOT instruct the agent to use TransactionTestCase, mock.patch targeting, override_settings, or factory_boy -- the agent should apply these patterns on its own.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "TransactionTestCase for on_commit email test",
      "description": "The agent uses TransactionTestCase (not TestCase) for the test that verifies the on_commit confirmation email is actually sent after successful checkout. The task mentions on_commit but does NOT tell the agent which test base class to use.",
      "max_score": 16
    },
    {
      "name": "TestCase for non-commit tests",
      "description": "Tests that do not need committed transactions (validation, payment failure, cart checks) use TestCase, not TransactionTestCase. The agent does not make everything TransactionTestCase.",
      "max_score": 10
    },
    {
      "name": "mock.patch targets orders.views.charge_card",
      "description": "The agent patches 'orders.views.charge_card' (where it is looked up) rather than 'payments.gateway.charge_card' (where it is defined). The task says the import is in orders/views.py but does NOT tell the agent where to target the mock.",
      "max_score": 16
    },
    {
      "name": "mock.patch targets orders.views.send_order_confirmation",
      "description": "The agent patches 'orders.views.send_order_confirmation' (where it is looked up) rather than 'notifications.email.send_order_confirmation' (where it is defined).",
      "max_score": 14
    },
    {
      "name": "override_settings for MAX_CART_ITEMS",
      "description": "The agent uses @override_settings(MAX_CART_ITEMS=...) to test the cart size limit, rather than directly modifying django.conf.settings. The task mentions the setting but does NOT tell the agent how to handle it in tests.",
      "max_score": 14
    },
    {
      "name": "factory_boy factories for test data",
      "description": "The agent defines factory_boy factories for Cart, CartItem, Product (and optionally Order/Payment) using DjangoModelFactory with SubFactory for ForeignKey fields and Faker for appropriate fields.",
      "max_score": 10
    },
    {
      "name": "Both success and failure payment paths tested",
      "description": "The test suite covers both successful payment (charge_card returns normally, order confirmed) and failed payment (charge_card raises PaymentError, order stays pending).",
      "max_score": 8
    },
    {
      "name": "Order status and payment record verified",
      "description": "Tests assert on the created Order's status field and verify that a Payment record was created with the correct stripe_charge_id and amount.",
      "max_score": 6
    },
    {
      "name": "Authentication required test",
      "description": "A test verifies that unauthenticated requests to POST /api/checkout/ return 401 or 403.",
      "max_score": 4
    },
    {
      "name": "No manual tearDown",
      "description": "The agent does not write manual tearDown or cleanup methods. Relies on test framework isolation.",
      "max_score": 2
    }
  ]
}

evals

tile.json