CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl-labs/django-testing

Write correct Django tests — TestCase vs TransactionTestCase, setUpTestData, factory-boy, assertNumQueries, mock.patch placement, and DRF APITestCase patterns

99

1.33x
Quality

99%

Does it follow best practices?

Impact

99%

1.33x

Average score across 2 eval scenarios

SecuritybySnyk

Passed

No known issues

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-1/

{
  "context": "Tests whether the agent proactively applies Django testing best practices when writing a blog API test suite. The task does NOT mention setUpTestData, assertNumQueries, factory_boy, or test isolation patterns -- the agent should apply these on its own.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "setUpTestData for read-only fixtures",
      "description": "The agent uses @classmethod setUpTestData(cls) for shared read-only test data such as users, tags, and published posts used across multiple read-only tests. Does NOT put all fixture creation in setUp when data is not mutated.",
      "max_score": 16
    },
    {
      "name": "assertNumQueries on list endpoints",
      "description": "The agent uses self.assertNumQueries() on at least the post list endpoint or tag list endpoint to verify constant query count and catch N+1 problems. The agent was NOT told to check query counts.",
      "max_score": 16
    },
    {
      "name": "factory_boy factories defined",
      "description": "The agent defines factory_boy factories using DjangoModelFactory with class Meta: model, SubFactory for ForeignKey fields (author, post), and factory.Faker for text fields. Factories are in a separate file or at the top of the test file.",
      "max_score": 14
    },
    {
      "name": "Factories used instead of objects.create",
      "description": "Test data setup calls the factory classes (e.g., PostFactory(), CommentFactory()) rather than Model.objects.create(). The agent was NOT told to use factories.",
      "max_score": 12
    },
    {
      "name": "TestCase as default base class",
      "description": "Test classes inherit from django.test.TestCase (or rest_framework.test.APITestCase) by default. TransactionTestCase is NOT used since the blog API has no on_commit hooks or signals requiring committed transactions.",
      "max_score": 8
    },
    {
      "name": "Happy path and error tests present",
      "description": "The test suite includes happy path tests (200/201), validation rejection (400 for empty/invalid data), 404 for missing posts, and persistence verification (create then retrieve).",
      "max_score": 10
    },
    {
      "name": "content_type on POST/PUT requests",
      "description": "POST and PUT requests to the Django test client include content_type='application/json' (or use DRF APIClient which handles this automatically).",
      "max_score": 6
    },
    {
      "name": "Authentication tests",
      "description": "Tests verify both authenticated and unauthenticated access -- e.g., unauthenticated POST returns 401/403, authenticated POST succeeds. Uses force_authenticate() or equivalent.",
      "max_score": 10
    },
    {
      "name": "No manual tearDown",
      "description": "The agent does not write manual tearDown or cleanup methods. Relies on TestCase transaction rollback for isolation.",
      "max_score": 4
    },
    {
      "name": "setUp only for mutable data",
      "description": "If both setUpTestData and setUp are used, setUp is reserved for data that individual tests will mutate (e.g., a post that will be deleted), while setUpTestData holds read-only shared data.",
      "max_score": 4
    }
  ]
}

evals

scenario-1

criteria.json

task.md

tile.json