tessl/pypi-lz4

LZ4 Bindings for Python providing high-performance lossless data compression with frame and block format support

—

Pending

Overview

Eval results

Files

Block Format

Name: tessl/pypi-lz4
Author: tessl

Block format compression provides low-level access to LZ4 block compression without container format overhead. This interface offers direct control over compression parameters and supports advanced features like dictionaries.

Capabilities

Core Compression Functions

Direct block compression and decompression without frame format overhead.

def compress(data, **kwargs) -> bytes:
    """
    Compress data as LZ4 block without container format.
    
    Args:
        data (bytes-like): Data to compress
        mode (str, optional): Compression mode ('default', 'fast', 'high_compression')
        acceleration (int, optional): Acceleration factor for fast mode (default: 1)
        compression_level (int, optional): Level for high_compression mode (1-12, default: 9)
        store_size (bool, optional): Store uncompressed size in output (default: True)
        return_bytearray (bool, optional): Return bytearray instead of bytes (default: False)
        dict (bytes, optional): Compression dictionary for improved ratio
    
    Returns:
        bytes: Compressed block data
    """

def decompress(data, **kwargs) -> bytes:
    """
    Decompress LZ4 block format data.
    
    Args:
        data (bytes-like): Compressed block data
        uncompressed_size (int, optional): Maximum output size hint (required if store_size=False was used)
        return_bytearray (bool, optional): Return bytearray instead of bytes (default: False)
        dict (bytes, optional): Decompression dictionary matching compression dictionary
    
    Returns:
        bytes: Decompressed data
    """

Exception Handling

Exception class for LZ4 block operations.

class LZ4BlockError(Exception):
    """Exception raised when LZ4 library operations fail."""

Compression Modes

Default Mode

Standard compression balancing speed and compression ratio.

import lz4.block

data = b"Hello, World!" * 1000
compressed = lz4.block.compress(data, mode='default')
decompressed = lz4.block.decompress(compressed)

Fast Mode

Optimized for compression speed with configurable acceleration.

import lz4.block

# Faster compression with acceleration
compressed = lz4.block.compress(
    data, 
    mode='fast', 
    acceleration=4  # Higher values = faster but less compression
)
decompressed = lz4.block.decompress(compressed)

High Compression Mode

Prioritizes compression ratio over speed with configurable levels.

import lz4.block

# Maximum compression
compressed = lz4.block.compress(
    data, 
    mode='high_compression', 
    compression_level=12  # 1-12, higher = better compression
)
decompressed = lz4.block.decompress(compressed)

Advanced Features

Dictionary Compression

Using dictionaries for improved compression of similar data.

import lz4.block

# Train dictionary on representative data
dictionary = b"common prefix data that appears frequently"

# Compress with dictionary
data = b"common prefix data with unique suffix"
compressed = lz4.block.compress(data, dict=dictionary)

# Decompress with same dictionary
decompressed = lz4.block.decompress(compressed, dict=dictionary)

Size Management

Control over size storage and output buffer management.

import lz4.block

# Compress without storing size (requires manual size tracking)
compressed = lz4.block.compress(data, store_size=False)

# Decompress with explicit size (required when store_size=False)
decompressed = lz4.block.decompress(
    compressed, 
    uncompressed_size=len(original_data)
)

# Use bytearray for reduced memory copying
compressed = lz4.block.compress(data, return_bytearray=True)
decompressed = lz4.block.decompress(compressed, return_bytearray=True)

Usage Examples

Basic Block Compression

import lz4.block

# Simple compression
data = b"Hello, World!" * 1000
compressed = lz4.block.compress(data)
decompressed = lz4.block.decompress(compressed)

print(f"Original size: {len(data)}")
print(f"Compressed size: {len(compressed)}")
print(f"Compression ratio: {len(data) / len(compressed):.2f}")

Performance Optimization

import lz4.block

# Fast compression for real-time applications
def fast_compress(data):
    return lz4.block.compress(
        data, 
        mode='fast', 
        acceleration=8,
        return_bytearray=True
    )

# High compression for storage
def archive_compress(data, dictionary=None):
    return lz4.block.compress(
        data,
        mode='high_compression',
        compression_level=12,
        dict=dictionary
    )

Error Handling

import lz4.block

try:
    # Attempt compression
    compressed = lz4.block.compress(data)
    decompressed = lz4.block.decompress(compressed)
    
except lz4.block.LZ4BlockError as e:
    print(f"LZ4 operation failed: {e}")
    # Handle compression/decompression failure
except Exception as e:
    print(f"Unexpected error: {e}")

Memory-Efficient Processing

import lz4.block

def process_large_data(data_chunks):
    """Process large data in chunks to minimize memory usage."""
    compressed_chunks = []
    
    for chunk in data_chunks:
        try:
            # Use bytearray to reduce copying
            compressed = lz4.block.compress(
                chunk, 
                mode='fast',
                return_bytearray=True
            )
            compressed_chunks.append(compressed)
            
        except lz4.block.LZ4BlockError:
            # Handle compression failure
            compressed_chunks.append(chunk)  # Store uncompressed
    
    return compressed_chunks

Install with Tessl CLI