tessl/pypi-zstandard

Zstandard bindings for Python providing high-performance compression and decompression operations

—

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Securityby

Pending

The risk profile of this skill

Overview

Eval results

Files

Zstandard

Name: tessl/pypi-zstandard
Author: tessl

Python bindings for the Zstandard compression library, providing high-performance lossless data compression and decompression. This library offers both simple one-shot functions and sophisticated streaming interfaces, supporting advanced features like dictionary compression, multi-threading, and customizable compression parameters while maintaining full compatibility with the standard Zstandard format.

Package Information

Package Name: zstandard
Language: Python
Installation: pip install zstandard

Core Imports

import zstandard

Common pattern for accessing main functionality:

import zstandard as zstd

Import specific components:

from zstandard import ZstdCompressor, ZstdDecompressor, compress, decompress

Basic Usage

import zstandard as zstd

# Simple one-shot compression and decompression
original_data = b"Hello, World! This is some data to compress."

# Compress data
compressed = zstd.compress(original_data, level=3)
print(f"Compressed size: {len(compressed)} bytes")

# Decompress data
decompressed = zstd.decompress(compressed)
print(f"Decompressed: {decompressed}")

# Using compressor and decompressor objects for better performance
compressor = zstd.ZstdCompressor(level=5)
decompressor = zstd.ZstdDecompressor()

# Multiple compressions with same compressor (more efficient)
data1 = b"First piece of data"
data2 = b"Second piece of data"

compressed1 = compressor.compress(data1)
compressed2 = compressor.compress(data2)

decompressed1 = decompressor.decompress(compressed1)
decompressed2 = decompressor.decompress(compressed2)

# File-like interface
with zstd.open('data.zst', 'wb') as f:
    f.write(b"Data to be compressed and written to file")

with zstd.open('data.zst', 'rb') as f:
    data = f.read()
    print(f"Read from file: {data}")

Architecture

The zstandard library is built around a multi-backend architecture that automatically selects the best available implementation:

C Extension Backend: High-performance native implementation (default for CPython)
CFFI Backend: Pure Python implementation with C bindings (default for PyPy)
Rust Backend: Rust-based implementation (experimental)

Core design patterns:

Compressor/Decompressor Objects: Reusable contexts for multiple operations
Streaming Interfaces: Memory-efficient processing of large data
Buffer Management: Advanced buffer types for zero-copy operations
Multi-threading Support: Parallel compression/decompression for improved performance

Capabilities

Simple Compression Operations

One-shot compression and decompression functions for basic use cases, plus file-like interface for easy integration with existing code.

def compress(data: bytes, level: int = 3) -> bytes: ...
def decompress(data: bytes, max_output_size: int = 0) -> bytes: ...
def open(filename, mode: str = "rb", **kwargs): ...

Simple Operations

Advanced Compression

Full-featured compression with customizable parameters, dictionary support, and streaming interfaces for high-performance applications.

class ZstdCompressor:
    def __init__(self, level: int = 3, **kwargs): ...
    def compress(self, data: bytes) -> bytes: ...
    def stream_writer(self, writer, **kwargs): ...
    def multi_compress_to_buffer(self, data, **kwargs): ...

class ZstdCompressionParameters:
    def __init__(self, **kwargs): ...
    @staticmethod
    def from_level(level: int, **kwargs): ...

Advanced Compression

Advanced Decompression

Sophisticated decompression with streaming support, frame analysis, and batch processing capabilities.

class ZstdDecompressor:
    def __init__(self, **kwargs): ...
    def decompress(self, data: bytes, **kwargs) -> bytes: ...
    def stream_reader(self, source, **kwargs): ...
    def multi_decompress_to_buffer(self, frames, **kwargs): ...

Advanced Decompression

Dictionary Compression

Training and using custom dictionaries for improved compression ratios on similar data.

class ZstdCompressionDict:
    def __init__(self, data: bytes, **kwargs): ...
    def dict_id(self) -> int: ...
    def precompute_compress(self, **kwargs): ...

def train_dictionary(dict_size: int, samples: list, **kwargs) -> ZstdCompressionDict: ...

Dictionary Compression

Buffer Operations

Advanced buffer management for zero-copy operations and efficient batch processing.

class BufferWithSegments:
    def __init__(self, data: bytes, segments: bytes): ...
    def segments(self): ...
    def tobytes(self) -> bytes: ...

class BufferWithSegmentsCollection:
    def __init__(self, *args): ...
    def size(self) -> int: ...

Buffer Operations

Frame Analysis

Utilities for analyzing zstd frames and extracting metadata without full decompression.

def frame_content_size(data: bytes) -> int: ...
def frame_header_size(data: bytes) -> int: ...
def get_frame_parameters(data: bytes) -> FrameParameters: ...

class FrameParameters:
    content_size: int
    window_size: int
    dict_id: int
    has_checksum: bool

Frame Analysis

Constants and Configuration

The library exports numerous constants for configuring compression parameters, strategies, and format options:

# Version information
__version__: str
backend: str
backend_features: set[str]
ZSTD_VERSION: tuple

# Compression levels and strategies
MAX_COMPRESSION_LEVEL: int
STRATEGY_FAST: int
STRATEGY_DFAST: int
STRATEGY_GREEDY: int
STRATEGY_LAZY: int
STRATEGY_LAZY2: int
STRATEGY_BTLAZY2: int
STRATEGY_BTOPT: int
STRATEGY_BTULTRA: int
STRATEGY_BTULTRA2: int

# Recommended buffer sizes
COMPRESSION_RECOMMENDED_INPUT_SIZE: int
COMPRESSION_RECOMMENDED_OUTPUT_SIZE: int
DECOMPRESSION_RECOMMENDED_INPUT_SIZE: int
DECOMPRESSION_RECOMMENDED_OUTPUT_SIZE: int

# Block size limits
BLOCKSIZELOG_MAX: int
BLOCKSIZE_MAX: int

# Compression parameter limits
WINDOWLOG_MIN: int
WINDOWLOG_MAX: int
CHAINLOG_MIN: int
CHAINLOG_MAX: int
HASHLOG_MIN: int
HASHLOG_MAX: int
MINMATCH_MIN: int
MINMATCH_MAX: int
SEARCHLOG_MIN: int
SEARCHLOG_MAX: int
SEARCHLENGTH_MIN: int
SEARCHLENGTH_MAX: int
TARGETLENGTH_MIN: int
TARGETLENGTH_MAX: int
LDM_MINMATCH_MIN: int
LDM_MINMATCH_MAX: int
LDM_BUCKETSIZELOG_MAX: int

# Format options
FORMAT_ZSTD1: int
FORMAT_ZSTD1_MAGICLESS: int

# Dictionary types
DICT_TYPE_AUTO: int
DICT_TYPE_RAWCONTENT: int
DICT_TYPE_FULLDICT: int

# Flush modes
FLUSH_BLOCK: int
FLUSH_FRAME: int

# Compression object flush modes
COMPRESSOBJ_FLUSH_FINISH: int
COMPRESSOBJ_FLUSH_BLOCK: int

# Content size indicators
CONTENTSIZE_UNKNOWN: int
CONTENTSIZE_ERROR: int

# Frame identification
FRAME_HEADER: bytes
MAGIC_NUMBER: int

Exception Handling

class ZstdError(Exception):
    """Base exception for zstandard-related errors."""

All zstandard operations may raise ZstdError or its subclasses for compression/decompression failures, invalid parameters, or corrupted data.

Workspace: tessl
Visibility: Public
Created: 7 months ago
Last updated: 2 months ago
Describes: pkg:pypi/zstandard@0.24.x
Publish Source: CLI
Badge