CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl-labs/structured-logging

Every backend service MUST use structured logging — pino/structlog/slog, JSON format, request IDs, proper log levels. console.log is never acceptable.

88

3.03x
Quality

85%

Does it follow best practices?

Impact

97%

3.03x

Average score across 4 eval scenarios

SecuritybySnyk

Passed

No known issues

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-3/

{
  "context": "Tests whether the agent proactively sets up structured logging when building a Python FastAPI service. The task describes only business requirements for a notification preferences API -- it never mentions logging, structlog, request IDs, or observability. A production-grade FastAPI service should use structured logging from the start.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Structured logger used (not print)",
      "description": "Uses structlog, python-json-logger, or Python's logging module with a JSON formatter for all logging. No bare print() statements for operational output in the application code.",
      "max_score": 15
    },
    {
      "name": "Request ID middleware",
      "description": "Middleware assigns a unique request ID to each request -- reads from x-request-id header if present, otherwise generates a UUID. The request ID is available in route handlers.",
      "max_score": 12
    },
    {
      "name": "Request ID in response header",
      "description": "The assigned request ID is written back to the response via the x-request-id header.",
      "max_score": 5
    },
    {
      "name": "Request ID propagated to log entries",
      "description": "Log entries include the request_id field -- achieved via structlog.contextvars, logging filters, or equivalent mechanism so all logs within a request carry the ID.",
      "max_score": 10
    },
    {
      "name": "Business events logged at info level",
      "description": "Key business events (preferences created, preferences updated, notification check performed) are logged at info level with structured fields like user_id and notification_type -- not string interpolation.",
      "max_score": 10
    },
    {
      "name": "404 not logged as error",
      "description": "Not-found (404) responses when preferences don't exist are logged at warn or info level -- NOT at error level.",
      "max_score": 8
    },
    {
      "name": "Per-request completion log",
      "description": "A middleware logs one structured line per completed request containing HTTP method, path, status code, and response duration.",
      "max_score": 8
    },
    {
      "name": "Startup logged with structured logger",
      "description": "Service startup is logged using the structured logger (not print).",
      "max_score": 7
    },
    {
      "name": "No sensitive data logged",
      "description": "User IDs are acceptable in logs, but no PII like email addresses or phone numbers should appear in log output.",
      "max_score": 7
    },
    {
      "name": "Log level configurable via environment variable",
      "description": "Logger level is set from an environment variable (e.g. LOG_LEVEL) with a sensible default.",
      "max_score": 7
    },
    {
      "name": "Context cleanup after request",
      "description": "If using structlog contextvars, context is properly cleaned up (clear_contextvars or unbind_contextvars) to prevent context leaking between requests.",
      "max_score": 6
    },
    {
      "name": "Structured fields not string interpolation",
      "description": "Log calls pass data as keyword arguments or dict fields rather than f-strings or string concatenation.",
      "max_score": 5
    }
  ]
}

evals

tile.json