CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-hishel

Persistent cache implementation for httpx and httpcore following RFC 9111 specification

74

1.48x
Overview
Eval results
Files

serializers.mddocs/

Serializers

Serialization formats for persisting HTTP request/response pairs with metadata. Hishel provides multiple serializers for different use cases, from binary Pickle format to human-readable JSON and YAML formats.

Capabilities

Base Serializer

Abstract base class defining the serializer interface for converting HTTP requests/responses to/from storage formats.

class BaseSerializer:
    def dumps(self, response: Response, request: Request, metadata: Metadata) -> str | bytes:
        """
        Serialize HTTP response, request, and metadata.
        
        Parameters:
        - response: HTTP response object
        - request: HTTP request object  
        - metadata: Cache metadata dictionary
        
        Returns:
        - Serialized data as string or bytes
        """
    
    def loads(self, data: str | bytes) -> tuple[Response, Request, Metadata]:
        """
        Deserialize data back to HTTP response, request, and metadata.
        
        Parameters:
        - data: Serialized data as string or bytes
        
        Returns:
        - Tuple of (Response, Request, Metadata)
        """
    
    @property
    def is_binary(self) -> bool:
        """Whether this serializer produces binary data"""

Pickle Serializer

Binary serializer using Python's pickle format for compact and fast serialization.

class PickleSerializer(BaseSerializer):
    def dumps(self, response: Response, request: Request, metadata: Metadata) -> bytes:
        """
        Serialize using pickle format.
        
        Parameters:
        - response: HTTP response object
        - request: HTTP request object
        - metadata: Cache metadata dictionary
        
        Returns:
        - Pickled data as bytes
        """
    
    def loads(self, data: bytes) -> tuple[Response, Request, Metadata]:
        """
        Deserialize pickle data.
        
        Parameters:
        - data: Pickled data as bytes
        
        Returns:
        - Tuple of (Response, Request, Metadata)
        """
    
    @property
    def is_binary(self) -> bool:
        """Returns True (pickle produces binary data)"""

Usage Examples:

import hishel

# Use pickle serializer for compact storage
serializer = hishel.PickleSerializer()

# Use with file storage
storage = hishel.FileStorage(serializer=serializer)
with hishel.CacheClient(storage=storage) as client:
    response = client.get("https://api.example.com/data")

# Pickle is fastest and most compact but not human-readable

JSON Serializer

Text-based serializer using JSON format for human-readable and cross-language compatible storage.

class JSONSerializer(BaseSerializer):
    def dumps(self, response: Response, request: Request, metadata: Metadata) -> str:
        """
        Serialize using JSON format.
        
        Parameters:
        - response: HTTP response object
        - request: HTTP request object
        - metadata: Cache metadata dictionary
        
        Returns:
        - JSON string with formatted data
        """
    
    def loads(self, data: str) -> tuple[Response, Request, Metadata]:
        """
        Deserialize JSON data.
        
        Parameters:
        - data: JSON string
        
        Returns:
        - Tuple of (Response, Request, Metadata)
        """
    
    @property
    def is_binary(self) -> bool:
        """Returns False (JSON produces text data)"""

Usage Examples:

import hishel

# Use JSON serializer (default for most storage backends)
serializer = hishel.JSONSerializer()

# JSON format is human-readable and debuggable
storage = hishel.FileStorage(serializer=serializer)
with hishel.CacheClient(storage=storage) as client:
    response = client.get("https://api.example.com/data")

# Cached files can be inspected and edited manually

YAML Serializer

Text-based serializer using YAML format for the most human-readable storage format.

class YAMLSerializer(BaseSerializer):
    def dumps(self, response: Response, request: Request, metadata: Metadata) -> str:
        """
        Serialize using YAML format.
        
        Parameters:
        - response: HTTP response object
        - request: HTTP request object
        - metadata: Cache metadata dictionary
        
        Returns:
        - YAML string with formatted data
        
        Raises:
        - RuntimeError: If PyYAML is not installed
        """
    
    def loads(self, data: str) -> tuple[Response, Request, Metadata]:
        """
        Deserialize YAML data.
        
        Parameters:
        - data: YAML string
        
        Returns:
        - Tuple of (Response, Request, Metadata)
        
        Raises:
        - RuntimeError: If PyYAML is not installed
        """
    
    @property
    def is_binary(self) -> bool:
        """Returns False (YAML produces text data)"""

Usage Examples:

import hishel

# YAML serializer requires pyyaml: pip install hishel[yaml]
serializer = hishel.YAMLSerializer()

# YAML is the most readable format for debugging
storage = hishel.FileStorage(serializer=serializer)
with hishel.CacheClient(storage=storage) as client:
    response = client.get("https://api.example.com/data")

Metadata Structure

Cache metadata contains information about cached responses:

class Metadata(TypedDict):
    number_of_uses: int      # How many times response was served from cache
    created_at: datetime     # When response was first cached
    cache_key: str          # Cache key for the cached response

Model Cloning

Utility function for cloning HTTP request/response objects for serialization:

def clone_model(model: Request | Response) -> Request | Response:
    """
    Clone HTTP request or response object for safe serialization.
    
    Parameters:
    - model: Request or Response object to clone
    
    Returns:
    - Cloned object with only serializable extensions
    
    Note: Only includes known safe extensions for serialization.
    """

Usage Examples:

from httpcore import Request, Response
import hishel

# Clone for safe serialization
request = Request(b"GET", b"https://api.example.com/data")
safe_request = hishel.clone_model(request)

response = Response(200, [(b"content-type", b"application/json")])
safe_response = hishel.clone_model(response)

Serializer Comparison

SerializerFormatSizeSpeedHuman ReadableCross-Language
PickleBinarySmallFastNoNo (Python only)
JSONTextMediumMediumYesYes
YAMLTextLargeSlowVery GoodYes

Custom Serializers

You can create custom serializers by inheriting from BaseSerializer:

import json
from typing import Union
from httpcore import Request, Response
import hishel

class CustomJSONSerializer(hishel.BaseSerializer):
    def dumps(self, response: Response, request: Request, metadata: hishel.Metadata) -> str:
        # Custom serialization logic
        data = {
            "response": {
                "status": response.status,
                "headers": dict(response.headers),
                "content": response.content.decode('utf-8', errors='ignore')
            },
            "request": {
                "method": request.method.decode(),
                "url": str(request.url)
            },
            "metadata": metadata
        }
        return json.dumps(data, indent=2)
    
    def loads(self, data: str) -> tuple[Response, Request, hishel.Metadata]:
        # Custom deserialization logic
        parsed = json.loads(data)
        # Reconstruct objects...
        return response, request, metadata
    
    @property
    def is_binary(self) -> bool:
        return False

# Use custom serializer
storage = hishel.FileStorage(serializer=CustomJSONSerializer())

Installation Requirements

YAML serializer requires additional dependency:

# Install with YAML support
pip install hishel[yaml]

# Or install PyYAML separately
pip install pyyaml

Install with Tessl CLI

npx tessl i tessl/pypi-hishel

docs

cache-clients.md

cache-controller.md

cache-transports.md

http-headers.md

index.md

serializers.md

storage-backends.md

testing-utilities.md

tile.json