CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl-labs/graceful-degradation

Every external call needs a timeout, every timeout needs a fallback — resilience patterns for HTTP, databases, and third-party services

88

4.72x
Quality

90%

Does it follow best practices?

Impact

85%

4.72x

Average score across 5 eval scenarios

SecuritybySnyk

Passed

No known issues

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-3/

{
  "context": "Tests whether the agent proactively applies graceful degradation when building a recommendation service that chains three API calls. The task describes a business requirement only -- no mention of timeouts, retries, circuit breakers, or fallbacks. The agent should recognize these are external service calls and add resilience patterns on its own.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "timeouts-on-all-service-calls",
      "description": "Every HTTP request to the User Profile API, Recommendation Engine API, and Product Catalog API includes an explicit timeout (AbortSignal.timeout, axios timeout, or equivalent). No unguarded fetch() calls.",
      "max_score": 15
    },
    {
      "name": "fallback-for-user-profile",
      "description": "When the User Profile API fails or times out, the code falls back to a cached profile, a default profile, or gracefully degrades (e.g., returns generic recommendations instead of personalized ones). Does not just crash or return 500.",
      "max_score": 15
    },
    {
      "name": "fallback-for-catalog-enrichment",
      "description": "When the Product Catalog API fails, the code returns recommendations with partial data (e.g., just product IDs without full details) rather than failing the entire request.",
      "max_score": 12
    },
    {
      "name": "retry-with-exponential-backoff",
      "description": "At least the recommendation engine or profile service call includes retry logic with exponential backoff (delays that increase: e.g., 1s, 2s, 4s) and jitter. Fixed delays or no backoff is a failure.",
      "max_score": 12
    },
    {
      "name": "only-transient-errors-retried",
      "description": "Retry logic distinguishes between transient errors (429, 502, 503, 504, network errors) and non-transient errors (400, 401, 404, 422). Non-transient errors are not retried.",
      "max_score": 10
    },
    {
      "name": "circuit-breaker-pattern",
      "description": "A circuit breaker is implemented for at least one of the three services, tracking failures and short-circuiting to fallback after a threshold is reached.",
      "max_score": 10
    },
    {
      "name": "per-dependency-timeout-values",
      "description": "Different services have different timeout values reflecting their characteristics (e.g., recommendation engine might need more time than profile lookup). Not all hardcoded to the same value.",
      "max_score": 8
    },
    {
      "name": "structured-failure-logging",
      "description": "Each caught dependency failure is logged with structured context: dependency name, error details, which fallback was used, and relevant request context (userId).",
      "max_score": 10
    },
    {
      "name": "error-handling-does-not-leak",
      "description": "Internal error details (stack traces, internal URLs, auth tokens) are not exposed in the API response. Errors returned to clients are sanitized.",
      "max_score": 8
    }
  ]
}

evals

tile.json