tessl/pypi-numexpr

Fast numerical expression evaluator for NumPy that accelerates array operations through optimized implementations and multi-threading

—

Pending

Overview

Eval results

Files

Compiled Expressions

Name: tessl/pypi-numexpr
Author: tessl

Pre-compilation of expressions for repeated evaluation with different data, providing optimal performance for expressions used multiple times. The NumExpr class allows creating reusable expression objects that avoid repeated parsing and compilation overhead.

Capabilities

NumExpr Class

Creates a pre-compiled expression object that can be executed multiple times with different variable values, eliminating parsing and compilation overhead for repeated use.

class NumExpr:
    """
    Pre-compiled expression object for repeated evaluation.
    
    This class represents a compiled mathematical expression that can be
    executed multiple times with different input data, providing optimal
    performance when the same expression structure is used repeatedly.
    """
    
    def __init__(self, ex, signature=(), sanitize=True, **kwargs):
        """
        Create a compiled expression object.
        
        Parameters:
        - ex (str): Mathematical expression string to compile
        - signature (tuple): Variable signature for type checking and optimization
        - sanitize (bool): Enable expression sanitization for security
        - **kwargs: Additional compilation options
        
        Raises:
        ValueError: If expression syntax is invalid
        """
    
    def run(self, *args, **kwargs):
        """
        Execute the compiled expression with provided arguments.
        
        Parameters:
        - *args: Positional arguments corresponding to variables in signature order
        - **kwargs: Named arguments for variable substitution
        
        Returns:
        ndarray: Result of expression evaluation
        
        Raises:
        ValueError: If argument types or shapes are incompatible
        """
    
    @property
    def signature(self):
        """Get the variable signature for this compiled expression."""
    
    @property 
    def input_names(self):
        """Get the input variable names for this expression."""

Usage Examples:

import numpy as np
import numexpr as ne

# Create a compiled expression
expr = ne.NumExpr("a * b + c")

# Execute with different data sets
a1, b1, c1 = np.random.random((3, 10000))
result1 = expr.run(a1, b1, c1)

a2, b2, c2 = np.random.random((3, 5000))  
result2 = expr.run(a2, b2, c2)

# Using keyword arguments
result3 = expr.run(a=a1*2, b=b1*3, c=c1/2)

# Complex mathematical expression
complex_expr = ne.NumExpr("sin(x) * exp(-y/scale) + sqrt(x*y)")
x = np.linspace(0, 10, 1000)
y = np.linspace(0, 5, 1000)
for scale in [1, 2, 5, 10]:
    result = complex_expr.run(x, y, scale)
    # Process result...

Advanced Usage Patterns

Type-Specific Compilation

# Specify variable types for optimized compilation
expr = ne.NumExpr("a * b + c", signature=(('a', 'double'), ('b', 'double'), ('c', 'double')))

# Mixed types
mixed_expr = ne.NumExpr("mask & (values > threshold)", 
                       signature=(('mask', 'bool'), ('values', 'double'), ('threshold', 'double')))

Memory-Efficient Patterns

# Pre-allocate output array for repeated use
expr = ne.NumExpr("a * 2 + b")
output = np.empty(10000)

for i in range(100):
    a = get_data_a(i)  # Get new data
    b = get_data_b(i)
    expr.run(a, b, out=output)  # Reuse output array
    process_result(output)

Conditional Compilation

# Compile different expressions based on runtime conditions
if use_complex_math:
    expr = ne.NumExpr("sqrt(real(z)**2 + imag(z)**2)")
else:
    expr = ne.NumExpr("sqrt(a**2 + b**2)")

# Use the appropriate compiled expression
result = expr.run(data)

Expression Composition

# Build complex expressions from components
base_expr = "a * b"
if apply_transformation:
    full_expr = f"log({base_expr} + 1)"
else:
    full_expr = base_expr

compiled = ne.NumExpr(full_expr)

Performance Considerations

When to Use Compiled Expressions

Ideal Cases:

Expressions evaluated many times (>10 iterations)
Complex mathematical expressions with multiple operations
Fixed expression structure with varying data
Performance-critical loops where compilation overhead matters

Less Beneficial:

Simple expressions evaluated once or twice
Very small arrays (< 1000 elements)
Expressions that change frequently

Memory and Threading

Compiled expressions automatically:

Use the current threading configuration (get_num_threads())
Respect VML settings for accelerated functions
Apply chunking strategies for cache optimization
Reuse internal memory allocations across runs

Compilation Time vs. Execution Time

Compilation overhead: ~0.1-1ms for typical expressions
Execution speedup: 2-15x for medium to large arrays
Break-even point: Usually 2-5 evaluations depending on array size and expression complexity

The NumExpr class is most effective when the same expression structure is used repeatedly with different data, amortizing the compilation cost across multiple evaluations.

Install with Tessl CLI