CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-etils

Collection of common python utils for machine learning and scientific computing workflows

Pending
Overview
Eval results
Files

numpy-utilities.mddocs/

NumPy Utilities (enp)

Enhanced NumPy utilities providing array specifications, compatibility layers, mathematical operations, and geometry utilities for scientific computing and machine learning workflows.

Capabilities

Array Specifications

Define and work with array shape and dtype specifications.

class ArraySpec:
    """
    Specification for array shape and data type.
    """
    def __init__(self, shape: tuple[int, ...], dtype: np.dtype) -> None: ...
    
    @property
    def shape(self) -> tuple[int, ...]: ...
    @property
    def dtype(self) -> np.dtype: ...

NumPy Module Wrapper

Enhanced NumPy module with additional utilities.

class NpModule:
    """
    NumPy module wrapper with enhanced functionality.
    """
    def __init__(self, module: ModuleType) -> None: ...

Array Validation and Processing

Functions for checking and normalizing arrays.

def check_and_normalize_arrays(
    fn: Callable | None = None,
    *,
    strict: bool = True
) -> Callable:
    """
    Decorator for functions that validates and normalizes array inputs.
    
    Args:
        fn: Function to decorate (when used without parentheses)
        strict: Whether to apply strict validation rules
        
    Returns:
        Decorated function that validates array inputs
        
    Example:
        @check_and_normalize_arrays
        def process_arrays(arr1, arr2):
            return arr1 + arr2
    """

def is_array_str(arr: np.ndarray) -> bool:
    """
    Check if array contains string data.
    
    Args:
        arr: Input array
        
    Returns:
        True if array contains strings, False otherwise
    """

def is_dtype_str(dtype: np.dtype) -> bool:
    """
    Check if dtype represents string data.
    
    Args:
        dtype: NumPy data type
        
    Returns:
        True if dtype is string type, False otherwise
    """

def normalize_bytes2str(arr: np.ndarray) -> np.ndarray:
    """
    Convert bytes arrays to string arrays.
    
    Args:
        arr: Array containing bytes
        
    Returns:
        Array with bytes converted to strings
    """

Shape Manipulation (Einops Integration)

Advanced array reshaping using einops patterns.

def flatten(arr: np.ndarray, pattern: str) -> np.ndarray:
    """
    Flatten array using einops pattern.
    
    Args:
        arr: Input array
        pattern: Einops pattern string (e.g., 'h w c -> (h w) c')
        
    Returns:
        Flattened array according to pattern
    """

def unflatten(arr: np.ndarray, pattern: str, **axes_lengths) -> np.ndarray:
    """
    Unflatten array using einops pattern.
    
    Args:
        arr: Input flattened array
        pattern: Einops pattern string (e.g., '(h w) c -> h w c')
        **axes_lengths: Lengths for axes (e.g., h=28, w=28)
        
    Returns:
        Unflattened array according to pattern
    """

Mathematical Operations

Enhanced mathematical functions and interpolation.

def interp(
    x: np.ndarray,
    xp: np.ndarray,
    fp: np.ndarray,
    **kwargs
) -> np.ndarray:
    """
    Enhanced interpolation function.
    
    Args:
        x: X-coordinates at which to evaluate interpolated values
        xp: X-coordinates of data points
        fp: Y-coordinates of data points
        **kwargs: Additional interpolation options
        
    Returns:
        Interpolated values
    """

Geometry Utilities

Vector and geometric operations for 3D mathematics.

def angle_between(v1: np.ndarray, v2: np.ndarray) -> float:
    """
    Calculate angle between two vectors.
    
    Args:
        v1: First vector
        v2: Second vector
        
    Returns:
        Angle between vectors in radians
    """

def batch_dot(a: np.ndarray, b: np.ndarray) -> np.ndarray:
    """
    Batch dot product of arrays.
    
    Args:
        a: First array batch
        b: Second array batch
        
    Returns:
        Array of dot products
    """

def project_onto_plane(
    vector: np.ndarray,
    plane_normal: np.ndarray
) -> np.ndarray:
    """
    Project vector onto a plane defined by its normal.
    
    Args:
        vector: Vector to project
        plane_normal: Normal vector of the plane
        
    Returns:
        Projected vector on the plane
    """

def project_onto_vector(
    vector: np.ndarray,
    target_vector: np.ndarray
) -> np.ndarray:
    """
    Project vector onto another vector.
    
    Args:
        vector: Vector to project
        target_vector: Target vector for projection
        
    Returns:
        Projected vector
    """

Constants and Utilities

Mathematical constants and lazy utilities.

tau: float  # Mathematical constant τ (2π)

dtypes: ModuleType  # Data type definitions module

def lazy() -> Any:
    """
    Lazy NumPy utilities for deferred loading.
    
    Returns:
        Lazy-loaded NumPy utilities
    """

Utility Modules

Additional modules for specific functionality.

typing: ModuleType  # NumPy typing utilities
compat: ModuleType  # Compatibility utilities
linalg: ModuleType  # Linear algebra utilities
testing: ModuleType  # Testing utilities (when pytest available)

Deprecated Functions

Legacy functions maintained for compatibility.

def get_np_module() -> ModuleType:
    """
    Get NumPy module (DEPRECATED).
    
    Returns:
        NumPy module
        
    Note:
        This function is deprecated. Use numpy directly.
    """

def is_array(obj: Any) -> bool:
    """
    Check if object is array (DEPRECATED).
    
    Args:
        obj: Object to check
        
    Returns:
        True if object is array-like
        
    Note:
        This function is deprecated. Use isinstance checks.
    """

Usage Examples

Array Specifications

from etils import enp
import numpy as np

# Create array specifications
spec = enp.ArraySpec(shape=(224, 224, 3), dtype=np.float32)
print(f"Shape: {spec.shape}, Dtype: {spec.dtype}")

# Use in function signatures
def process_image(image: np.ndarray) -> np.ndarray:
    assert image.shape == spec.shape
    assert image.dtype == spec.dtype
    return image * 2.0

Array Validation

from etils import enp
import numpy as np

# Validate and normalize multiple arrays
arrays = [
    [1, 2, 3],                    # List
    np.array([4, 5, 6]),         # NumPy array
    (7, 8, 9)                    # Tuple
]

normalized = enp.check_and_normalize_arrays(*arrays)
# Result: List of numpy arrays

# Check array properties
str_array = np.array(['hello', 'world'])
is_string = enp.is_array_str(str_array)  # True

dtype_is_string = enp.is_dtype_str(str_array.dtype)  # True

Shape Manipulation

from etils import enp
import numpy as np

# Create sample image data
image = np.random.rand(28, 28, 3)  # Height x Width x Channels

# Flatten for neural network input
flattened = enp.flatten(image, 'h w c -> (h w c)')
# Result: Shape (2352,) = 28 * 28 * 3

# Unflatten back to original shape
restored = enp.unflatten(flattened, '(h w c) -> h w c', h=28, w=28, c=3)
# Result: Original (28, 28, 3) shape

# Batch operations
batch_images = np.random.rand(32, 28, 28, 3)  # Batch x H x W x C
batch_flat = enp.flatten(batch_images, 'b h w c -> b (h w c)')
# Result: Shape (32, 2352)

Mathematical Operations

from etils import enp
import numpy as np

# Interpolation
x = np.linspace(0, 10, 11)
y = np.sin(x)
x_new = np.linspace(0, 10, 101)
y_interp = enp.interp(x_new, x, y)

# Use mathematical constants
circumference = 2 * np.pi * radius  # Standard approach
circumference = enp.tau * radius    # Using tau constant

Geometry Operations

from etils import enp
import numpy as np

# Vector operations
v1 = np.array([1, 0, 0])
v2 = np.array([0, 1, 0])

# Calculate angle between vectors
angle = enp.angle_between(v1, v2)  # π/2 radians (90 degrees)

# Batch dot products
batch_a = np.random.rand(100, 3)
batch_b = np.random.rand(100, 3)
dot_products = enp.batch_dot(batch_a, batch_b)  # Shape: (100,)

# Projections
vector = np.array([1, 1, 1])
plane_normal = np.array([0, 0, 1])  # XY plane
projected = enp.project_onto_plane(vector, plane_normal)
# Result: [1, 1, 0] (projected onto XY plane)

target = np.array([1, 0, 0])  # X-axis
vector_proj = enp.project_onto_vector(vector, target)
# Result: [1, 0, 0] (component along X-axis)

String and Bytes Handling

from etils import enp
import numpy as np

# Convert bytes to strings
byte_array = np.array([b'hello', b'world'])
string_array = enp.normalize_bytes2str(byte_array)
# Result: array(['hello', 'world'], dtype='<U5')

# Check for string content
mixed_array = np.array(['text', 123])  # Mixed types
has_strings = enp.is_array_str(mixed_array)

Install with Tessl CLI

npx tessl i tessl/pypi-etils

docs

application-framework.md

array-types.md

colab-integration.md

dataclass-enhancements.md

index.md

numpy-utilities.md

path-operations.md

python-utilities.md

tree-manipulation.md

tile.json