or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

index.md
tile.json

tessl/pypi-crc32c

A Python package implementing the CRC32C checksum algorithm in hardware and software

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/crc32c@2.7.x

To install, run

npx @tessl/cli install tessl/pypi-crc32c@2.7.0

index.mddocs/

CRC32C

A Python package implementing the CRC32C checksum algorithm in hardware and software. It automatically chooses between hardware-accelerated implementations (using Intel SSE 4.2 CRC32C instructions on x86 CPUs, and crc32* instructions on ARMv8 CPUs) and optimized software fallbacks when hardware support is unavailable.

Package Information

  • Package Name: crc32c
  • Language: Python
  • Installation: pip install crc32c

Core Imports

import crc32c

Basic Usage

import crc32c

# Basic checksum calculation
checksum = crc32c.crc32c(b'hello world')
print(checksum)  # 3381945770

# Incremental checksum calculation
crc = crc32c.crc32c(b'hello')
final_crc = crc32c.crc32c(b' world', value=crc)
print(final_crc)  # 3381945770

# Using hash object interface
hasher = crc32c.CRC32CHash()
hasher.update(b'hello')
hasher.update(b' world')
print(hasher.checksum)  # 3381945770
print(hasher.digest())   # b'\xc9\x94e\xaa'
print(hasher.hexdigest())  # c99465aa

Capabilities

CRC32C Function

Calculate CRC32C checksum incrementally with configurable GIL release behavior.

def crc32c(data, value=0, gil_release_mode=-1):
    """
    Calculate CRC32C checksum incrementally.
    
    Parameters:
    - data: Buffer (bytes-like object) to checksum
    - value: int, initial checksum value (default: 0)
    - gil_release_mode: int, GIL release behavior:
        * Negative: Only release GIL when data >= 32KiB (default: -1)
        * 0: Never release the GIL
        * Positive: Always release the GIL
    
    Returns:
    int: The calculated CRC32C checksum
    """

CRC32 Function (Deprecated)

Legacy function name for CRC32C calculation. Use crc32c instead as this will be removed in future versions.

def crc32(data, value=0, gil_release_mode=-1):
    """
    Calculate CRC32C checksum incrementally (deprecated).
    
    Deprecated: Use crc32c instead. Will be removed in future versions.
    
    Parameters:
    - data: Buffer (bytes-like object) to checksum
    - value: int, initial checksum value (default: 0)  
    - gil_release_mode: int, GIL release behavior (same as crc32c)
    
    Returns:
    int: The calculated CRC32C checksum
    """

Hash Object Interface

Object-oriented interface modeled after Python's hashlib for CRC32C computation.

class CRC32CHash:
    """
    Hash object class for CRC32C computation following hashlib interface.
    """
    
    def __init__(self, data=b"", gil_release_mode=-1):
        """
        Initialize hash object with optional initial data.
        
        Parameters:
        - data: bytes-like object, initial data to hash (default: empty bytes)
        - gil_release_mode: int, GIL release behavior for checksum calculations (default: -1)
        """
    
    def update(self, data):
        """
        Update hash object with bytes-like object.
        
        Parameters:
        - data: bytes-like object to add to hash
        
        Returns:
        None
        """
    
    def digest(self):
        """
        Return digest as bytes object.
        
        Returns:
        bytes: 4-byte big-endian digest
        """
    
    def hexdigest(self):
        """
        Return digest as hexadecimal string.
        
        Returns:
        str: Hexadecimal representation of digest
        """
    
    def copy(self):
        """
        Return copy of hash object.
        
        Returns:
        CRC32CHash: Copy of current hash object state
        """
    
    @property
    def digest_size(self):
        """
        Size of resulting hash in bytes.
        
        Returns:
        int: Always 4 for CRC32C
        """
    
    @property
    def block_size(self):
        """
        Internal block size of hash algorithm in bytes.
        
        Returns:
        int: Always 1 for CRC32C
        """
    
    @property
    def name(self):
        """
        Canonical name of hash algorithm.
        
        Returns:
        str: Always "crc32c"
        """
    
    @property
    def checksum(self):
        """
        Current checksum value (non-standard hashlib extension).
        
        Returns:
        int: Current checksum as integer
        """

Module Constants

Runtime information about implementation and platform characteristics.

hardware_based: bool
"""
Indicates whether hardware-based implementation is in use.
True if using hardware acceleration, False if using software fallback.
"""

big_endian: int
"""
Indicates platform endianness.
1 if platform is big-endian, 0 if little-endian.
"""

Benchmarking Utilities

Performance testing functionality available as a submodule.

import crc32c.benchmark

def run(size, iterations):
    """
    Run benchmark with specified parameters.
    
    Parameters:
    - size: int, amount of bytes to checksum
    - iterations: int, number of iterations to run
    
    Returns:
    tuple[float, int]: (duration_seconds, total_evaluations)
    """

def main():
    """
    Command-line benchmark interface with argument parsing.
    
    Returns:
    None
    """

The benchmark module can be executed directly:

python -m crc32c.benchmark --size 104857600 --iterations 10

Environment Variables

Package behavior can be controlled via environment variables:

CRC32C_SW_MODE

Controls software implementation preference:

  • auto: Use software fallback if no hardware support (default behavior, will be discontinued)
  • force: Force software implementation regardless of hardware support
  • none: Raise RuntimeError if no hardware support found
  • unset: Use software fallback if no hardware support

CRC32C_SKIP_HW_PROBE

For testing purposes:

  • 1: Skip hardware capability detection
  • unset: Normal hardware detection

Types

from typing import Union
from typing_extensions import Buffer

Buffer = Union[bytes, bytearray, memoryview, ...]
"""
Type alias for bytes-like objects that support the buffer protocol.
Includes bytes, bytearray, memoryview, and other objects implementing __buffer__.
"""

Error Handling

class RuntimeError(Exception):
    """
    Raised when no hardware support is available and CRC32C_SW_MODE is set to "none".
    """

class RuntimeWarning(Warning):
    """
    Warning issued when no hardware support is available and CRC32C_SW_MODE is set to "none".
    """

class DeprecationWarning(Warning):
    """
    Warning issued when using the deprecated crc32 function.
    """

Platform Support

  • Hardware Acceleration:
    • Intel x86/x64 with SSE 4.2 CRC32C instructions
    • ARMv8 with crc32* instructions
  • Software Fallback: Intel slice-by-8 algorithm on all platforms
  • Threading: Configurable GIL release modes for optimal multi-threading performance
  • Binary Wheels: Available for major platforms/architectures via PyPI