Structured Logging — Non-Negotiable for Every Backend Service

Any time you create or modify a backend service, API, or background worker, you MUST set up structured logging. There is no exception. console.log, print, and fmt.Println are never acceptable in production code.

This is not optional. This is not something to add later. Structured logging is part of building the application, just like error handling and input validation.

What You Must Do — Every Time

When you build any backend service (API, worker, CLI tool, background processor), always:

1. Install a structured logging library — pino for Node.js, structlog for Python, slog for Go. Never use console.log/print/fmt.Println.
2. Output JSON — all logs must be JSON-formatted for production log aggregation.
3. Use log levels correctly — error, warn, info, debug each have specific meanings (see below).
4. Add request ID middleware (for HTTP services) — every request gets a unique ID propagated through all log entries.
5. Log request completion — one structured log line per request with method, path, status code, and duration.
6. Log business events — at info level with structured fields (not string interpolation).
7. Log startup and shutdown — with the structured logger, not console.log.
8. Never log sensitive data — no passwords, tokens, API keys, or PII in log output.
9. Configure log level via environment variable — LOG_LEVEL env var with info as default.

Log Levels — Use Them Correctly

Critical rules:

• 404 Not Found is warn, NOT error — it's an expected condition
• Insufficient resources / validation failures are warn, NOT error
• info should tell the business story of what happened
• debug is for development only — too noisy for production

Request ID Pattern

Every HTTP service must assign and propagate request IDs for traceability.

Node.js (Express + pino)

import pino from 'pino';
import { randomUUID } from 'crypto';

const logger = pino({ level: process.env.LOG_LEVEL || 'info' });

// Middleware: assign request ID and child logger
app.use((req, res, next) => {
  req.id = req.headers['x-request-id'] as string || randomUUID();
  res.setHeader('x-request-id', req.id);
  req.log = logger.child({ request_id: req.id });
  next();
});

// Request completion logging middleware
app.use((req, res, next) => {
  const start = Date.now();
  res.on('finish', () => {
    req.log.info({
      method: req.method,
      path: req.path,
      status: res.statusCode,
      duration_ms: Date.now() - start,
    }, "request_completed");
  });
  next();
});

// In route handlers — use req.log, structured fields, correct levels
router.post("/api/orders", (req, res) => {
  req.log.info({ customer_id: body.customer_id }, "creating_order");
  const order = orderService.create(body);
  req.log.info({ order_id: order.id, total_cents: order.total }, "order_created");
  res.status(201).json({ data: order });
});

Python (FastAPI + structlog)

import structlog
import uuid

logger = structlog.get_logger()

@app.middleware("http")
async def add_request_id(request, call_next):
    request_id = request.headers.get("x-request-id", str(uuid.uuid4()))
    structlog.contextvars.clear_contextvars()
    structlog.contextvars.bind_contextvars(request_id=request_id)
    response = await call_next(request)
    response.headers["x-request-id"] = request_id
    structlog.contextvars.unbind_contextvars("request_id")
    return response

@app.post("/api/orders")
async def create_order(body: CreateOrderRequest):
    logger.info("creating_order", customer_id=body.customer_id)
    order = order_service.create(body)
    logger.info("order_created", order_id=order.id, total_cents=order.total_cents)
    return {"data": order}

Go (slog)

import "log/slog"

func requestIDMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        requestID := r.Header.Get("X-Request-ID")
        if requestID == "" {
            requestID = uuid.NewString()
        }
        w.Header().Set("X-Request-ID", requestID)
        logger := slog.With("request_id", requestID)
        ctx := context.WithValue(r.Context(), "logger", logger)
        next.ServeHTTP(w, r.WithContext(ctx))
    })
}

Background Workers and Job Processors

For non-HTTP services (queue consumers, cron jobs, background workers):

1. Use job/task ID instead of request ID — correlate all logs for a single job execution
2. Log job lifecycle — job_started, job_completed, job_failed with duration
3. Log startup and shutdown of the worker process
4. Include queue/topic metadata in log context

const logger = pino({ level: process.env.LOG_LEVEL || 'info' });

async function processJob(job: Job) {
  const jobLog = logger.child({ job_id: job.id, queue: job.queue });
  jobLog.info({ type: job.type }, "job_started");
  try {
    const result = await executeJob(job);
    jobLog.info({ duration_ms: result.duration }, "job_completed");
  } catch (err) {
    jobLog.error({ err, type: job.type }, "job_failed");
    throw err;
  }
}

What to Log vs. What Never to Log

Always log

• Request received and completed (method, path, status, duration)
• Business events (order created, payment processed, user action)
• Errors with context (what failed, relevant IDs, stack in structured field)
• Startup and shutdown
• Job/task lifecycle (started, completed, failed with duration)

Never log

• Passwords, password hashes, tokens, API keys
• Full request/response bodies (may contain PII)
• Credit card numbers, SSNs, or other financial PII
• Health check requests (too noisy — skip in request logging middleware)

Error Logging

Log errors with full context in structured fields — never embed stack traces in the message string:

// GOOD — structured error with context
req.log.error({ err, order_id: orderId, customer_id: customerId }, "order_creation_failed");

// BAD — unstructured string
console.error(`Error creating order: ${err.message} ${err.stack}`);

Checklist — Apply to Every Backend Service

• Structured logger installed (pino / structlog / slog — never console.log/print)
• JSON output format configured
• Log levels used correctly (error only for unexpected failures)
• Request ID or Job ID assigned and propagated to all log entries
• One log line per request with method, path, status, duration
• Business events logged at info level with structured fields
• No sensitive data in logs (passwords, tokens, PII)
• Errors logged with context (relevant IDs, error object in structured field)
• Startup and shutdown logged
• Log level configurable via LOG_LEVEL environment variable

Verifiers

• request-id-propagation — Assign and propagate request IDs through log entries
• use-structured-logger — Use a structured logging library instead of console.log/print