CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-crc32c

A Python package implementing the CRC32C checksum algorithm in hardware and software

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Overview
Eval results
Files

index.mddocs/

CRC32C

A Python package implementing the CRC32C checksum algorithm in hardware and software. It automatically chooses between hardware-accelerated implementations (using Intel SSE 4.2 CRC32C instructions on x86 CPUs, and crc32* instructions on ARMv8 CPUs) and optimized software fallbacks when hardware support is unavailable.

Package Information

  • Package Name: crc32c
  • Language: Python
  • Installation: pip install crc32c

Core Imports

import crc32c

Basic Usage

import crc32c

# Basic checksum calculation
checksum = crc32c.crc32c(b'hello world')
print(checksum)  # 3381945770

# Incremental checksum calculation
crc = crc32c.crc32c(b'hello')
final_crc = crc32c.crc32c(b' world', value=crc)
print(final_crc)  # 3381945770

# Using hash object interface
hasher = crc32c.CRC32CHash()
hasher.update(b'hello')
hasher.update(b' world')
print(hasher.checksum)  # 3381945770
print(hasher.digest())   # b'\xc9\x94e\xaa'
print(hasher.hexdigest())  # c99465aa

Capabilities

CRC32C Function

Calculate CRC32C checksum incrementally with configurable GIL release behavior.

def crc32c(data, value=0, gil_release_mode=-1):
    """
    Calculate CRC32C checksum incrementally.
    
    Parameters:
    - data: Buffer (bytes-like object) to checksum
    - value: int, initial checksum value (default: 0)
    - gil_release_mode: int, GIL release behavior:
        * Negative: Only release GIL when data >= 32KiB (default: -1)
        * 0: Never release the GIL
        * Positive: Always release the GIL
    
    Returns:
    int: The calculated CRC32C checksum
    """

CRC32 Function (Deprecated)

Legacy function name for CRC32C calculation. Use crc32c instead as this will be removed in future versions.

def crc32(data, value=0, gil_release_mode=-1):
    """
    Calculate CRC32C checksum incrementally (deprecated).
    
    Deprecated: Use crc32c instead. Will be removed in future versions.
    
    Parameters:
    - data: Buffer (bytes-like object) to checksum
    - value: int, initial checksum value (default: 0)  
    - gil_release_mode: int, GIL release behavior (same as crc32c)
    
    Returns:
    int: The calculated CRC32C checksum
    """

Hash Object Interface

Object-oriented interface modeled after Python's hashlib for CRC32C computation.

class CRC32CHash:
    """
    Hash object class for CRC32C computation following hashlib interface.
    """
    
    def __init__(self, data=b"", gil_release_mode=-1):
        """
        Initialize hash object with optional initial data.
        
        Parameters:
        - data: bytes-like object, initial data to hash (default: empty bytes)
        - gil_release_mode: int, GIL release behavior for checksum calculations (default: -1)
        """
    
    def update(self, data):
        """
        Update hash object with bytes-like object.
        
        Parameters:
        - data: bytes-like object to add to hash
        
        Returns:
        None
        """
    
    def digest(self):
        """
        Return digest as bytes object.
        
        Returns:
        bytes: 4-byte big-endian digest
        """
    
    def hexdigest(self):
        """
        Return digest as hexadecimal string.
        
        Returns:
        str: Hexadecimal representation of digest
        """
    
    def copy(self):
        """
        Return copy of hash object.
        
        Returns:
        CRC32CHash: Copy of current hash object state
        """
    
    @property
    def digest_size(self):
        """
        Size of resulting hash in bytes.
        
        Returns:
        int: Always 4 for CRC32C
        """
    
    @property
    def block_size(self):
        """
        Internal block size of hash algorithm in bytes.
        
        Returns:
        int: Always 1 for CRC32C
        """
    
    @property
    def name(self):
        """
        Canonical name of hash algorithm.
        
        Returns:
        str: Always "crc32c"
        """
    
    @property
    def checksum(self):
        """
        Current checksum value (non-standard hashlib extension).
        
        Returns:
        int: Current checksum as integer
        """

Module Constants

Runtime information about implementation and platform characteristics.

hardware_based: bool
"""
Indicates whether hardware-based implementation is in use.
True if using hardware acceleration, False if using software fallback.
"""

big_endian: int
"""
Indicates platform endianness.
1 if platform is big-endian, 0 if little-endian.
"""

Benchmarking Utilities

Performance testing functionality available as a submodule.

import crc32c.benchmark

def run(size, iterations):
    """
    Run benchmark with specified parameters.
    
    Parameters:
    - size: int, amount of bytes to checksum
    - iterations: int, number of iterations to run
    
    Returns:
    tuple[float, int]: (duration_seconds, total_evaluations)
    """

def main():
    """
    Command-line benchmark interface with argument parsing.
    
    Returns:
    None
    """

The benchmark module can be executed directly:

python -m crc32c.benchmark --size 104857600 --iterations 10

Environment Variables

Package behavior can be controlled via environment variables:

CRC32C_SW_MODE

Controls software implementation preference:

  • auto: Use software fallback if no hardware support (default behavior, will be discontinued)
  • force: Force software implementation regardless of hardware support
  • none: Raise RuntimeError if no hardware support found
  • unset: Use software fallback if no hardware support

CRC32C_SKIP_HW_PROBE

For testing purposes:

  • 1: Skip hardware capability detection
  • unset: Normal hardware detection

Types

from typing import Union
from typing_extensions import Buffer

Buffer = Union[bytes, bytearray, memoryview, ...]
"""
Type alias for bytes-like objects that support the buffer protocol.
Includes bytes, bytearray, memoryview, and other objects implementing __buffer__.
"""

Error Handling

class RuntimeError(Exception):
    """
    Raised when no hardware support is available and CRC32C_SW_MODE is set to "none".
    """

class RuntimeWarning(Warning):
    """
    Warning issued when no hardware support is available and CRC32C_SW_MODE is set to "none".
    """

class DeprecationWarning(Warning):
    """
    Warning issued when using the deprecated crc32 function.
    """

Platform Support

  • Hardware Acceleration:
    • Intel x86/x64 with SSE 4.2 CRC32C instructions
    • ARMv8 with crc32* instructions
  • Software Fallback: Intel slice-by-8 algorithm on all platforms
  • Threading: Configurable GIL release modes for optimal multi-threading performance
  • Binary Wheels: Available for major platforms/architectures via PyPI

Install with Tessl CLI

npx tessl i tessl/pypi-crc32c

docs

index.md

tile.json