CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl-labs/api-idempotency

Idempotent API design — safe retries for POST endpoints, idempotency keys,

93

10.00x
Quality

90%

Does it follow best practices?

Impact

100%

10.00x

Average score across 4 eval scenarios

SecuritybySnyk

Passed

No known issues

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-1/

{
  "context": "Tests whether the agent proactively adds idempotency protection to a user registration endpoint with both client and server components. The task says nothing about duplicates or retries. A skilled agent should recognize that POST /api/users needs protection against duplicate account creation and double-click submission.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Client sends idempotency key",
      "description": "The registration form JavaScript generates a unique key (UUID or similar) and sends it as an 'Idempotency-Key' header or body field in the POST request",
      "max_score": 16
    },
    {
      "name": "Server checks for duplicate requests",
      "description": "The server has a mechanism to detect duplicate registration requests — either via idempotency key lookup, database unique constraint on a client-supplied key, or deduplication middleware",
      "max_score": 16
    },
    {
      "name": "Submit button disabled during request",
      "description": "The registration button is disabled while the POST request is in flight to prevent double-click duplicate submissions",
      "max_score": 14
    },
    {
      "name": "Button re-enabled in finally block",
      "description": "The button re-enable logic is in a finally block (or equivalent) so the form remains usable after errors",
      "max_score": 10
    },
    {
      "name": "Database unique constraint on idempotency key",
      "description": "The users table has a UNIQUE constraint on an idempotency key column (separate from email uniqueness) that prevents duplicate insertions from retried requests",
      "max_score": 14
    },
    {
      "name": "Duplicate handled gracefully",
      "description": "When a duplicate registration is detected, the server returns the existing user record (or a success response) rather than an error — making retries safe",
      "max_score": 14
    },
    {
      "name": "Non-duplicate errors not swallowed",
      "description": "Error handling only suppresses the specific duplicate/constraint error — other errors are properly propagated",
      "max_score": 8
    },
    {
      "name": "5xx errors allow retry",
      "description": "If idempotency caching is present, server errors do not get cached — the client can retry after a 5xx",
      "max_score": 8
    }
  ]
}

evals

scenario-1

criteria.json

task.md

tile.json