or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

compiled-expressions.mdexpression-analysis.mdexpression-evaluation.mdindex.mdthreading-performance.mdvml-integration.md
tile.json

tessl/pypi-numexpr

Fast numerical expression evaluator for NumPy that accelerates array operations through optimized implementations and multi-threading

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/numexpr@2.11.x

To install, run

npx @tessl/cli install tessl/pypi-numexpr@2.11.0

index.mddocs/

NumExpr

A fast numerical expression evaluator for NumPy that accelerates array operations through optimized C++ implementations and multi-threading capabilities. NumExpr parses mathematical expressions into custom op-codes executed by an integrated virtual machine, avoiding intermediate memory allocations and enabling substantial performance improvements over pure NumPy operations.

Package Information

  • Package Name: numexpr
  • Package Type: pypi
  • Language: Python (with C++ extensions)
  • Installation: pip install numexpr

Core Imports

import numexpr as ne

For specific functions:

from numexpr import evaluate, re_evaluate, NumExpr, set_num_threads, get_num_threads

Basic Usage

import numpy as np
import numexpr as ne

# Create sample arrays
a = np.arange(1e6)  # Large arrays show better performance gains
b = np.arange(1e6)

# Simple expression evaluation
result = ne.evaluate("a + 1")

# Complex expressions with multiple operations
result = ne.evaluate("a * b - 4.1 * a > 2.5 * b")

# Using variables from local scope
x = 2.5
y = 4.1
result = ne.evaluate("a * b - y * a > x * b")

# With explicit variable dictionary
result = ne.evaluate("a * b - y * a > x * b", 
                     local_dict={'a': a, 'b': b, 'x': 2.5, 'y': 4.1})

# Pre-compile expressions for repeated evaluation
compiled_expr = ne.NumExpr("a * b + c")
result1 = compiled_expr.run(a, b, c=np.ones(len(a)))
result2 = compiled_expr.run(a*2, b*3, c=np.zeros(len(a)))

Architecture

NumExpr uses a multi-stage compilation and execution pipeline:

  • Expression Parser: Converts string expressions into Abstract Syntax Trees (AST)
  • Type Inference: Determines result types and validates operations
  • Code Generation: Translates AST into custom virtual machine opcodes
  • Virtual Machine: Executes opcodes on chunked array data across multiple threads
  • Memory Management: Uses cache-friendly chunking to avoid intermediate allocations

This architecture enables 0.95x to 15x performance improvements over NumPy by:

  • Eliminating temporary array allocations
  • Optimizing cache utilization through data chunking
  • Parallelizing computation across CPU cores
  • Leveraging Intel VML when available for transcendental functions

Capabilities

Expression Evaluation

Core functionality for evaluating mathematical expressions on NumPy arrays with performance optimization and multi-threading support.

def evaluate(ex, local_dict=None, global_dict=None, out=None, order='K', casting='safe', **kwargs):
    """
    Evaluate a mathematical expression on arrays.
    
    Parameters:
    - ex (str): Mathematical expression string
    - local_dict (dict, optional): Local variable bindings
    - global_dict (dict, optional): Global variable bindings  
    - out (ndarray, optional): Output array for result
    - order (str): Memory layout order ('K', 'A', 'C', 'F')
    - casting (str): Casting behavior ('no', 'equiv', 'safe', 'same_kind', 'unsafe')
    
    Returns:
    ndarray: Result of expression evaluation
    """

def re_evaluate(local_dict=None, **kwargs):
    """
    Re-evaluate the previous expression with new variable values.
    
    Parameters:
    - local_dict (dict, optional): New local variable bindings
    
    Returns:
    ndarray: Result of re-evaluation
    """

Expression Evaluation

Compiled Expressions

Pre-compilation of expressions for repeated evaluation with different data, providing optimal performance for expressions used multiple times.

class NumExpr:
    """
    Pre-compiled expression object for repeated evaluation.
    """
    def __init__(self, ex, signature=(), sanitize=True, **kwargs):
        """
        Create a compiled expression.
        
        Parameters:
        - ex (str): Mathematical expression string
        - signature (tuple): Variable signature for type checking
        - sanitize (bool): Enable expression sanitization
        """
    
    def run(self, *args, **kwargs):
        """Execute the compiled expression with provided arguments."""

Compiled Expressions

Threading and Performance Control

Configuration of multi-threading behavior and performance optimization settings for CPU-intensive computations.

def set_num_threads(nthreads):
    """
    Set the number of threads for operations.
    
    Parameters:
    - nthreads (int): Number of threads to use
    
    Returns:
    int: Previous thread count
    """

def get_num_threads():
    """
    Get the current number of threads in use.
    
    Returns:
    int: Current thread count
    """

def detect_number_of_cores():
    """
    Detect the number of CPU cores available.
    
    Returns:
    int: Number of detected cores
    """

Threading and Performance

VML Integration

Integration with Intel's Vector Math Library (VML) for hardware-accelerated transcendental functions when available.

def get_vml_version():
    """
    Get the VML/MKL library version.
    
    Returns:
    str or None: VML version string if available
    """

def set_vml_accuracy_mode(mode):
    """
    Set VML accuracy mode.
    
    Parameters:
    - mode (str): 'high', 'low', 'fast', or None
    
    Returns:
    str: Previous accuracy mode
    """

def set_vml_num_threads(nthreads):
    """Set number of threads for VML operations."""

VML Integration

Expression Analysis and Debugging

Tools for analyzing, validating, and debugging expressions including disassembly of compiled expressions.

def validate(ex, local_dict=None, global_dict=None, out=None, order='K', casting='safe', **kwargs):
    """
    Validate an expression without evaluating it.
    
    Parameters:
    - ex (str): Expression to validate
    - Additional parameters same as evaluate()
    
    Returns:
    tuple: (result_type, result_shape) information
    """

def disassemble(nex):
    """
    Disassemble a NumExpr object showing internal opcodes.
    
    Parameters:
    - nex (NumExpr): Compiled expression object
    
    Returns:
    str: Human-readable disassembly
    """

Expression Analysis

Testing and Diagnostics

Development and debugging utilities for testing NumExpr functionality and examining system capabilities.

def print_versions():
    """
    Print the versions of software that numexpr relies on.
    
    Displays version information for NumExpr, NumPy, Python, and system
    details useful for debugging and support.
    """

def test(verbosity=1):
    """
    Run all the tests in the test suite.
    
    Parameters:
    - verbosity (int): Test output verbosity level
    
    Returns:
    TestResult: Test execution results
    """

Module Constants

# Version information
__version__: str  # Package version string

# Threading constants  
MAX_THREADS: int  # Maximum supported threads
__BLOCK_SIZE1__: int  # Virtual machine block size

# Feature availability
use_vml: bool  # Whether VML support is available

# Runtime state
ncores: int  # Detected number of CPU cores
nthreads: int  # Current configured thread count

# Expression objects and utilities
E: object  # Expression builder object for creating expressions

Expression Language

NumExpr supports a comprehensive mathematical expression language including:

Arithmetic Operations: +, -, *, /, %, ** Comparison Operations: ==, !=, <, <=, >, >=
Logical Operations: & (and), | (or), ~ (not) Mathematical Functions: sin, cos, tan, arcsin, arccos, arctan, sinh, cosh, tanh, arcsinh, arccosh, arctanh, exp, expm1, log, log1p, log10, sqrt, abs, ceil, floor, arctan2, fmod Complex Functions: real, imag, complex, conjugate String Operations: String comparison and basic operations

Data Types: bool, int, long, float, double, complex, str

All operations support NumPy broadcasting semantics and work with NumPy array types.