CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-ormsgpack

Fast, correct Python msgpack library supporting dataclasses, datetimes, and numpy

Pending
Overview
Eval results
Files

index.mddocs/

ormsgpack

A fast msgpack serialization library for Python with native support for complex data types including dataclasses, datetime objects, numpy arrays, and pydantic models. Built with Rust for optimal performance, ormsgpack provides a simple API with extensive customization options for serialization behavior.

Package Information

  • Package Name: ormsgpack
  • Language: Python (with Rust backend)
  • Installation: pip install ormsgpack

Core Imports

import ormsgpack

Common pattern for accessing options:

import ormsgpack
from ormsgpack import OPT_NAIVE_UTC, OPT_SERIALIZE_NUMPY

Basic Usage

import ormsgpack
import datetime
import numpy as np

# Basic serialization and deserialization
data = {"key": "value", "number": 42, "list": [1, 2, 3]}
packed = ormsgpack.packb(data)
unpacked = ormsgpack.unpackb(packed)
print(unpacked)  # {'key': 'value', 'number': 42, 'list': [1, 2, 3]}

# Serialization with options for complex types
complex_data = {
    "timestamp": datetime.datetime(2023, 1, 15, 10, 30, 45),
    "array": np.array([[1, 2], [3, 4]]),
    "status": "active"
}

# Use options to handle datetime and numpy serialization
packed = ormsgpack.packb(
    complex_data, 
    option=ormsgpack.OPT_NAIVE_UTC | ormsgpack.OPT_SERIALIZE_NUMPY
)
unpacked = ormsgpack.unpackb(packed)
print(unpacked)

Capabilities

Serialization

Converts Python objects to msgpack format with extensive customization options for handling various data types.

def packb(
    __obj: Any,
    default: Optional[Callable[[Any], Any]] = None,
    option: Optional[int] = None,
) -> bytes:
    """
    Serialize Python objects to msgpack format.
    
    Parameters:
    - __obj: Any Python object to serialize (positional-only)
    - default: Optional callable for handling non-natively supported types
    - option: Optional integer bitmask of serialization options
    
    Returns:
    bytes: The serialized msgpack data
    
    Raises:
    MsgpackEncodeError: If serialization fails due to:
                       - Unsupported object type
                       - Invalid UTF-8 in str objects
                       - Non-str/bytes dict keys (without OPT_NON_STR_KEYS)
                       - Circular references
                       - Unsupported tzinfo on datetime objects
                       - Default callable recursion > 254 levels
    """

Deserialization

Converts msgpack data back to Python objects with optional extension type handling.

def unpackb(
    __obj: Union[bytes, bytearray, memoryview],
    /,
    ext_hook: Optional[Callable[[int, bytes], Any]] = None,
    option: Optional[int] = None,
) -> Any:
    """
    Deserialize msgpack data to Python objects.
    
    Parameters:
    - __obj: bytes, bytearray, or memoryview containing msgpack data (positional-only)
    - ext_hook: Optional callable for handling extension types during deserialization
    - option: Optional integer bitmask of deserialization options
             Supported options: OPT_DATETIME_AS_TIMESTAMP_EXT, OPT_NON_STR_KEYS
    
    Returns:
    Any: The deserialized Python object
    
    Raises:
    MsgpackDecodeError: If deserialization fails due to:
                       - Invalid msgpack format
                       - Unsupported msgpack type
    """

Extension Types

Handle custom data types through msgpack extension mechanism.

class Ext:
    """
    Represents a msgpack extension type with a tag and data.
    """
    def __init__(self, tag: int, data: bytes) -> None:
        """
        Create an extension type.
        
        Parameters:
        - tag: Integer tag identifying the extension type (0-127)
        - data: bytes containing the extension data
        """
    
    tag: int  # The extension type tag (0-127)
    data: bytes  # The extension data as bytes

Serialization Options

Options can be combined using bitwise OR (|) operator to customize serialization behavior:

option = ormsgpack.OPT_NAIVE_UTC | ormsgpack.OPT_SERIALIZE_NUMPY | ormsgpack.OPT_SORT_KEYS

DateTime Options

OPT_NAIVE_UTC: int  # Treat naive datetime objects as UTC (packb only)
OPT_OMIT_MICROSECONDS: int  # Omit microseconds from datetime serialization (packb only)
OPT_DATETIME_AS_TIMESTAMP_EXT: int  # Serialize/deserialize datetime as msgpack timestamp extension (packb/unpackb)
OPT_UTC_Z: int  # Use 'Z' suffix for UTC timezone in datetime strings (packb only)

Collection Options

OPT_NON_STR_KEYS: int  # Allow non-string keys in dictionaries (packb/unpackb)
OPT_SORT_KEYS: int  # Sort dictionary keys in serialized output (packb only)

Type-Specific Serialization

OPT_SERIALIZE_NUMPY: int  # Enable native serialization of numpy arrays (packb only)
OPT_SERIALIZE_PYDANTIC: int  # Enable native serialization of pydantic models (packb only)

Passthrough Options

These options disable custom serialization for specific types, passing them through as-is (packb only):

OPT_PASSTHROUGH_BIG_INT: int  # Pass through large integers without custom serialization
OPT_PASSTHROUGH_DATACLASS: int  # Pass through dataclass objects without custom serialization
OPT_PASSTHROUGH_DATETIME: int  # Pass through datetime objects without custom serialization
OPT_PASSTHROUGH_ENUM: int  # Pass through enum objects without custom serialization
OPT_PASSTHROUGH_SUBCLASS: int  # Pass through subclass instances without custom serialization
OPT_PASSTHROUGH_TUPLE: int  # Pass through tuple objects without custom serialization
OPT_PASSTHROUGH_UUID: int  # Pass through UUID objects without custom serialization

Exception Types

class MsgpackDecodeError(ValueError):
    """
    Raised when msgpack deserialization fails.
    
    Inherits from ValueError. Occurs when the input data is not valid msgpack
    format or contains unsupported msgpack types.
    """

class MsgpackEncodeError(TypeError):
    """
    Raised when msgpack serialization fails.
    
    Inherits from TypeError. Provides detailed error messages describing
    the object type that caused the serialization failure, with format:
    'Type is not msgpack serializable: <type_name>'
    """

Supported Data Types

ormsgpack natively supports serialization of:

  • Basic types: None, bool, int, float, str, bytes
  • Collections: list, tuple, dict
  • Advanced types: dataclass, datetime, date, time, enum, UUID
  • Third-party types: numpy arrays (with OPT_SERIALIZE_NUMPY), pydantic models (with OPT_SERIALIZE_PYDANTIC)
  • Extension types: Custom types via Ext class and ext_hook

Advanced Usage Examples

Custom Type Handling

import ormsgpack
from decimal import Decimal

def decimal_handler(obj):
    if isinstance(obj, Decimal):
        return str(obj)
    raise TypeError

# Serialize with custom handler
data = {"price": Decimal("19.99")}
packed = ormsgpack.packb(data, default=decimal_handler)
unpacked = ormsgpack.unpackb(packed)

Extension Types

import ormsgpack

# Create extension type
ext_data = ormsgpack.Ext(42, b"custom_data")
packed = ormsgpack.packb(ext_data)

# Handle extension during unpacking
def ext_handler(tag, data):
    if tag == 42:
        return f"Custom: {data.decode()}"
    return ormsgpack.Ext(tag, data)

unpacked = ormsgpack.unpackb(packed, ext_hook=ext_handler)

Dataclass Serialization

import ormsgpack
from dataclasses import dataclass

@dataclass
class Person:
    name: str
    age: int

person = Person("Alice", 30)
packed = ormsgpack.packb(person)  # Default behavior serializes as dict
unpacked = ormsgpack.unpackb(packed)  # Returns {'name': 'Alice', 'age': 30}

Version Information

__version__: str  # Version string of the ormsgpack library

Install with Tessl CLI

npx tessl i tessl/pypi-ormsgpack

docs

index.md

tile.json