0
# NumExpr
1

2
A fast numerical expression evaluator for NumPy that accelerates array operations through optimized C++ implementations and multi-threading capabilities. NumExpr parses mathematical expressions into custom op-codes executed by an integrated virtual machine, avoiding intermediate memory allocations and enabling substantial performance improvements over pure NumPy operations.
3

4
## Package Information
5

6
- **Package Name**: numexpr
7
- **Package Type**: pypi
8
- **Language**: Python (with C++ extensions)
9
- **Installation**: `pip install numexpr`
10

11
## Core Imports
12

13
```python
14
import numexpr as ne
15
```
16

17
For specific functions:
18

19
```python
20
from numexpr import evaluate, re_evaluate, NumExpr, set_num_threads, get_num_threads
21
```
22

23
## Basic Usage
24

25
```python
26
import numpy as np
27
import numexpr as ne
28

29
# Create sample arrays
30
a = np.arange(1e6)  # Large arrays show better performance gains
31
b = np.arange(1e6)
32

33
# Simple expression evaluation
34
result = ne.evaluate("a + 1")
35

36
# Complex expressions with multiple operations
37
result = ne.evaluate("a * b - 4.1 * a > 2.5 * b")
38

39
# Using variables from local scope
40
x = 2.5
41
y = 4.1
42
result = ne.evaluate("a * b - y * a > x * b")
43

44
# With explicit variable dictionary
45
result = ne.evaluate("a * b - y * a > x * b", 
46
                     local_dict={'a': a, 'b': b, 'x': 2.5, 'y': 4.1})
47

48
# Pre-compile expressions for repeated evaluation
49
compiled_expr = ne.NumExpr("a * b + c")
50
result1 = compiled_expr.run(a, b, c=np.ones(len(a)))
51
result2 = compiled_expr.run(a*2, b*3, c=np.zeros(len(a)))
52
```
53

54
## Architecture
55

56
NumExpr uses a multi-stage compilation and execution pipeline:
57

58
- **Expression Parser**: Converts string expressions into Abstract Syntax Trees (AST)
59
- **Type Inference**: Determines result types and validates operations
60
- **Code Generation**: Translates AST into custom virtual machine opcodes
61
- **Virtual Machine**: Executes opcodes on chunked array data across multiple threads
62
- **Memory Management**: Uses cache-friendly chunking to avoid intermediate allocations
63

64
This architecture enables 0.95x to 15x performance improvements over NumPy by:
65
- Eliminating temporary array allocations
66
- Optimizing cache utilization through data chunking
67
- Parallelizing computation across CPU cores
68
- Leveraging Intel VML when available for transcendental functions
69

70
## Capabilities
71

72
### Expression Evaluation
73

74
Core functionality for evaluating mathematical expressions on NumPy arrays with performance optimization and multi-threading support.
75

76
```python { .api }
77
def evaluate(ex, local_dict=None, global_dict=None, out=None, order='K', casting='safe', **kwargs):
78
    """
79
    Evaluate a mathematical expression on arrays.
80
    
81
    Parameters:
82
    - ex (str): Mathematical expression string
83
    - local_dict (dict, optional): Local variable bindings
84
    - global_dict (dict, optional): Global variable bindings  
85
    - out (ndarray, optional): Output array for result
86
    - order (str): Memory layout order ('K', 'A', 'C', 'F')
87
    - casting (str): Casting behavior ('no', 'equiv', 'safe', 'same_kind', 'unsafe')
88
    
89
    Returns:
90
    ndarray: Result of expression evaluation
91
    """
92

93
def re_evaluate(local_dict=None, **kwargs):
94
    """
95
    Re-evaluate the previous expression with new variable values.
96
    
97
    Parameters:
98
    - local_dict (dict, optional): New local variable bindings
99
    
100
    Returns:
101
    ndarray: Result of re-evaluation
102
    """
103
```
104

105
[Expression Evaluation](./expression-evaluation.md)
106

107
### Compiled Expressions
108

109
Pre-compilation of expressions for repeated evaluation with different data, providing optimal performance for expressions used multiple times.
110

111
```python { .api }
112
class NumExpr:
113
    """
114
    Pre-compiled expression object for repeated evaluation.
115
    """
116
    def __init__(self, ex, signature=(), sanitize=True, **kwargs):
117
        """
118
        Create a compiled expression.
119
        
120
        Parameters:
121
        - ex (str): Mathematical expression string
122
        - signature (tuple): Variable signature for type checking
123
        - sanitize (bool): Enable expression sanitization
124
        """
125
    
126
    def run(self, *args, **kwargs):
127
        """Execute the compiled expression with provided arguments."""
128
```
129

130
[Compiled Expressions](./compiled-expressions.md)
131

132
### Threading and Performance Control
133

134
Configuration of multi-threading behavior and performance optimization settings for CPU-intensive computations.
135

136
```python { .api }
137
def set_num_threads(nthreads):
138
    """
139
    Set the number of threads for operations.
140
    
141
    Parameters:
142
    - nthreads (int): Number of threads to use
143
    
144
    Returns:
145
    int: Previous thread count
146
    """
147

148
def get_num_threads():
149
    """
150
    Get the current number of threads in use.
151
    
152
    Returns:
153
    int: Current thread count
154
    """
155

156
def detect_number_of_cores():
157
    """
158
    Detect the number of CPU cores available.
159
    
160
    Returns:
161
    int: Number of detected cores
162
    """
163
```
164

165
[Threading and Performance](./threading-performance.md)
166

167
### VML Integration
168

169
Integration with Intel's Vector Math Library (VML) for hardware-accelerated transcendental functions when available.
170

171
```python { .api }
172
def get_vml_version():
173
    """
174
    Get the VML/MKL library version.
175
    
176
    Returns:
177
    str or None: VML version string if available
178
    """
179

180
def set_vml_accuracy_mode(mode):
181
    """
182
    Set VML accuracy mode.
183
    
184
    Parameters:
185
    - mode (str): 'high', 'low', 'fast', or None
186
    
187
    Returns:
188
    str: Previous accuracy mode
189
    """
190

191
def set_vml_num_threads(nthreads):
192
    """Set number of threads for VML operations."""
193
```
194

195
[VML Integration](./vml-integration.md)
196

197
### Expression Analysis and Debugging
198

199
Tools for analyzing, validating, and debugging expressions including disassembly of compiled expressions.
200

201
```python { .api }
202
def validate(ex, local_dict=None, global_dict=None, out=None, order='K', casting='safe', **kwargs):
203
    """
204
    Validate an expression without evaluating it.
205
    
206
    Parameters:
207
    - ex (str): Expression to validate
208
    - Additional parameters same as evaluate()
209
    
210
    Returns:
211
    tuple: (result_type, result_shape) information
212
    """
213

214
def disassemble(nex):
215
    """
216
    Disassemble a NumExpr object showing internal opcodes.
217
    
218
    Parameters:
219
    - nex (NumExpr): Compiled expression object
220
    
221
    Returns:
222
    str: Human-readable disassembly
223
    """
224
```
225

226
[Expression Analysis](./expression-analysis.md)
227

228
### Testing and Diagnostics
229

230
Development and debugging utilities for testing NumExpr functionality and examining system capabilities.
231

232
```python { .api }
233
def print_versions():
234
    """
235
    Print the versions of software that numexpr relies on.
236
    
237
    Displays version information for NumExpr, NumPy, Python, and system
238
    details useful for debugging and support.
239
    """
240

241
def test(verbosity=1):
242
    """
243
    Run all the tests in the test suite.
244
    
245
    Parameters:
246
    - verbosity (int): Test output verbosity level
247
    
248
    Returns:
249
    TestResult: Test execution results
250
    """
251
```
252

253
## Module Constants
254

255
```python { .api }
256
# Version information
257
__version__: str  # Package version string
258

259
# Threading constants  
260
MAX_THREADS: int  # Maximum supported threads
261
__BLOCK_SIZE1__: int  # Virtual machine block size
262

263
# Feature availability
264
use_vml: bool  # Whether VML support is available
265

266
# Runtime state
267
ncores: int  # Detected number of CPU cores
268
nthreads: int  # Current configured thread count
269

270
# Expression objects and utilities
271
E: object  # Expression builder object for creating expressions
272
```
273

274
## Expression Language
275

276
NumExpr supports a comprehensive mathematical expression language including:
277

278
**Arithmetic Operations**: `+`, `-`, `*`, `/`, `%`, `**`
279
**Comparison Operations**: `==`, `!=`, `<`, `<=`, `>`, `>=`  
280
**Logical Operations**: `&` (and), `|` (or), `~` (not)
281
**Mathematical Functions**: `sin`, `cos`, `tan`, `arcsin`, `arccos`, `arctan`, `sinh`, `cosh`, `tanh`, `arcsinh`, `arccosh`, `arctanh`, `exp`, `expm1`, `log`, `log1p`, `log10`, `sqrt`, `abs`, `ceil`, `floor`, `arctan2`, `fmod`
282
**Complex Functions**: `real`, `imag`, `complex`, `conjugate`
283
**String Operations**: String comparison and basic operations
284

285
**Data Types**: `bool`, `int`, `long`, `float`, `double`, `complex`, `str`
286

287
All operations support NumPy broadcasting semantics and work with NumPy array types.