Every external call needs a timeout, every timeout needs a fallback — resilience patterns for HTTP, databases, and third-party services
88
90%
Does it follow best practices?
Impact
85%
4.72xAverage score across 5 eval scenarios
Passed
No known issues
{
"context": "Tests whether the agent proactively applies graceful degradation when building a recommendation service that chains three API calls. The task describes a business requirement only -- no mention of timeouts, retries, circuit breakers, or fallbacks. The agent should recognize these are external service calls and add resilience patterns on its own.",
"type": "weighted_checklist",
"checklist": [
{
"name": "timeouts-on-all-service-calls",
"description": "Every HTTP request to the User Profile API, Recommendation Engine API, and Product Catalog API includes an explicit timeout (AbortSignal.timeout, axios timeout, or equivalent). No unguarded fetch() calls.",
"max_score": 15
},
{
"name": "fallback-for-user-profile",
"description": "When the User Profile API fails or times out, the code falls back to a cached profile, a default profile, or gracefully degrades (e.g., returns generic recommendations instead of personalized ones). Does not just crash or return 500.",
"max_score": 15
},
{
"name": "fallback-for-catalog-enrichment",
"description": "When the Product Catalog API fails, the code returns recommendations with partial data (e.g., just product IDs without full details) rather than failing the entire request.",
"max_score": 12
},
{
"name": "retry-with-exponential-backoff",
"description": "At least the recommendation engine or profile service call includes retry logic with exponential backoff (delays that increase: e.g., 1s, 2s, 4s) and jitter. Fixed delays or no backoff is a failure.",
"max_score": 12
},
{
"name": "only-transient-errors-retried",
"description": "Retry logic distinguishes between transient errors (429, 502, 503, 504, network errors) and non-transient errors (400, 401, 404, 422). Non-transient errors are not retried.",
"max_score": 10
},
{
"name": "circuit-breaker-pattern",
"description": "A circuit breaker is implemented for at least one of the three services, tracking failures and short-circuiting to fallback after a threshold is reached.",
"max_score": 10
},
{
"name": "per-dependency-timeout-values",
"description": "Different services have different timeout values reflecting their characteristics (e.g., recommendation engine might need more time than profile lookup). Not all hardcoded to the same value.",
"max_score": 8
},
{
"name": "structured-failure-logging",
"description": "Each caught dependency failure is logged with structured context: dependency name, error details, which fallback was used, and relevant request context (userId).",
"max_score": 10
},
{
"name": "error-handling-does-not-leak",
"description": "Internal error details (stack traces, internal URLs, auth tokens) are not exposed in the API response. Errors returned to clients are sanitized.",
"max_score": 8
}
]
}