HTTP Snapshot Fetcher

A small utility that fetches HTTP or HTTPS responses while merging query parameters, respecting basic limits, and reporting the final response details.

Capabilities

Query merging and headers

• Given a base URL that already has query parameters, it appends provided key/value pairs without dropping existing ones and includes a default user agent header when none is supplied. @test
• When explicit headers are provided, they override defaults and appear in the outbound request. @test

Size guard via HEAD

• When a maximum byte budget is provided, it performs a HEAD request first and aborts with an error before downloading if Content-Length exceeds the limit. @test

Redirect handling

• It follows up to three HTTP redirects, returns the final URL, and raises an error when the redirect limit is exceeded. @test

Timeout handling

• It honors a configurable timeout in seconds and raises a timeout error when the server does not respond within that window. @test

Implementation

@generates

API

from dataclasses import dataclass
from typing import Mapping, Optional


class SnapshotError(Exception):
    """Raised for any fetch-related failures."""


@dataclass
class SnapshotResult:
    final_url: str
    status: int
    headers: Mapping[str, str]
    body: str


def fetch_snapshot(
    url: str,
    query: Optional[Mapping[str, str]] = None,
    headers: Optional[Mapping[str, str]] = None,
    max_bytes: Optional[int] = None,
    timeout: float = 5.0,
    redirect_limit: int = 3,
) -> SnapshotResult:
    """
    Fetches text content over HTTP/HTTPS while merging query parameters, enforcing an optional size guard, following redirects, and respecting timeouts.
    Raises SnapshotError on failures.
    """

Dependencies { .dependencies }

python-stdlib-networking { .dependency }

Provides built-in HTTP clients, URL parsing, and socket timeout support for network I/O.

@satisfied-by