Fast numerical expression evaluator for NumPy that accelerates array operations through optimized implementations and multi-threading
npx @tessl/cli install tessl/pypi-numexpr@2.11.0A fast numerical expression evaluator for NumPy that accelerates array operations through optimized C++ implementations and multi-threading capabilities. NumExpr parses mathematical expressions into custom op-codes executed by an integrated virtual machine, avoiding intermediate memory allocations and enabling substantial performance improvements over pure NumPy operations.
pip install numexprimport numexpr as neFor specific functions:
from numexpr import evaluate, re_evaluate, NumExpr, set_num_threads, get_num_threadsimport numpy as np
import numexpr as ne
# Create sample arrays
a = np.arange(1e6) # Large arrays show better performance gains
b = np.arange(1e6)
# Simple expression evaluation
result = ne.evaluate("a + 1")
# Complex expressions with multiple operations
result = ne.evaluate("a * b - 4.1 * a > 2.5 * b")
# Using variables from local scope
x = 2.5
y = 4.1
result = ne.evaluate("a * b - y * a > x * b")
# With explicit variable dictionary
result = ne.evaluate("a * b - y * a > x * b",
local_dict={'a': a, 'b': b, 'x': 2.5, 'y': 4.1})
# Pre-compile expressions for repeated evaluation
compiled_expr = ne.NumExpr("a * b + c")
result1 = compiled_expr.run(a, b, c=np.ones(len(a)))
result2 = compiled_expr.run(a*2, b*3, c=np.zeros(len(a)))NumExpr uses a multi-stage compilation and execution pipeline:
This architecture enables 0.95x to 15x performance improvements over NumPy by:
Core functionality for evaluating mathematical expressions on NumPy arrays with performance optimization and multi-threading support.
def evaluate(ex, local_dict=None, global_dict=None, out=None, order='K', casting='safe', **kwargs):
"""
Evaluate a mathematical expression on arrays.
Parameters:
- ex (str): Mathematical expression string
- local_dict (dict, optional): Local variable bindings
- global_dict (dict, optional): Global variable bindings
- out (ndarray, optional): Output array for result
- order (str): Memory layout order ('K', 'A', 'C', 'F')
- casting (str): Casting behavior ('no', 'equiv', 'safe', 'same_kind', 'unsafe')
Returns:
ndarray: Result of expression evaluation
"""
def re_evaluate(local_dict=None, **kwargs):
"""
Re-evaluate the previous expression with new variable values.
Parameters:
- local_dict (dict, optional): New local variable bindings
Returns:
ndarray: Result of re-evaluation
"""Pre-compilation of expressions for repeated evaluation with different data, providing optimal performance for expressions used multiple times.
class NumExpr:
"""
Pre-compiled expression object for repeated evaluation.
"""
def __init__(self, ex, signature=(), sanitize=True, **kwargs):
"""
Create a compiled expression.
Parameters:
- ex (str): Mathematical expression string
- signature (tuple): Variable signature for type checking
- sanitize (bool): Enable expression sanitization
"""
def run(self, *args, **kwargs):
"""Execute the compiled expression with provided arguments."""Configuration of multi-threading behavior and performance optimization settings for CPU-intensive computations.
def set_num_threads(nthreads):
"""
Set the number of threads for operations.
Parameters:
- nthreads (int): Number of threads to use
Returns:
int: Previous thread count
"""
def get_num_threads():
"""
Get the current number of threads in use.
Returns:
int: Current thread count
"""
def detect_number_of_cores():
"""
Detect the number of CPU cores available.
Returns:
int: Number of detected cores
"""Integration with Intel's Vector Math Library (VML) for hardware-accelerated transcendental functions when available.
def get_vml_version():
"""
Get the VML/MKL library version.
Returns:
str or None: VML version string if available
"""
def set_vml_accuracy_mode(mode):
"""
Set VML accuracy mode.
Parameters:
- mode (str): 'high', 'low', 'fast', or None
Returns:
str: Previous accuracy mode
"""
def set_vml_num_threads(nthreads):
"""Set number of threads for VML operations."""Tools for analyzing, validating, and debugging expressions including disassembly of compiled expressions.
def validate(ex, local_dict=None, global_dict=None, out=None, order='K', casting='safe', **kwargs):
"""
Validate an expression without evaluating it.
Parameters:
- ex (str): Expression to validate
- Additional parameters same as evaluate()
Returns:
tuple: (result_type, result_shape) information
"""
def disassemble(nex):
"""
Disassemble a NumExpr object showing internal opcodes.
Parameters:
- nex (NumExpr): Compiled expression object
Returns:
str: Human-readable disassembly
"""Development and debugging utilities for testing NumExpr functionality and examining system capabilities.
def print_versions():
"""
Print the versions of software that numexpr relies on.
Displays version information for NumExpr, NumPy, Python, and system
details useful for debugging and support.
"""
def test(verbosity=1):
"""
Run all the tests in the test suite.
Parameters:
- verbosity (int): Test output verbosity level
Returns:
TestResult: Test execution results
"""# Version information
__version__: str # Package version string
# Threading constants
MAX_THREADS: int # Maximum supported threads
__BLOCK_SIZE1__: int # Virtual machine block size
# Feature availability
use_vml: bool # Whether VML support is available
# Runtime state
ncores: int # Detected number of CPU cores
nthreads: int # Current configured thread count
# Expression objects and utilities
E: object # Expression builder object for creating expressionsNumExpr supports a comprehensive mathematical expression language including:
Arithmetic Operations: +, -, *, /, %, **
Comparison Operations: ==, !=, <, <=, >, >=
Logical Operations: & (and), | (or), ~ (not)
Mathematical Functions: sin, cos, tan, arcsin, arccos, arctan, sinh, cosh, tanh, arcsinh, arccosh, arctanh, exp, expm1, log, log1p, log10, sqrt, abs, ceil, floor, arctan2, fmod
Complex Functions: real, imag, complex, conjugate
String Operations: String comparison and basic operations
Data Types: bool, int, long, float, double, complex, str
All operations support NumPy broadcasting semantics and work with NumPy array types.