0
# Testing Framework
1
2
Comprehensive testing utilities with detailed error reporting, specialized assertion helpers for different data types, and utilities for testing exceptions and warnings. Designed for use in notebooks and scripts with clear, informative error messages.
3
4
## Capabilities
5
6
### Core Testing Functions
7
8
Essential testing functions that provide clear error messages and handle various data types including arrays, iterables, and custom objects.
9
10
```python { .api }
11
def test_eq(a, b):
12
"""
13
Test that a equals b with detailed error reporting.
14
15
Uses fastcore's enhanced equality checking that handles:
16
- numpy arrays with array_equal
17
- pandas DataFrames with df_equal
18
- nested structures with proper comparison
19
- Custom objects with appropriate equality methods
20
21
Parameters:
22
- a: First value to compare
23
- b: Second value to compare
24
25
Raises:
26
AssertionError: If values are not equal, with detailed comparison
27
"""
28
29
def test_eq_type(a, b):
30
"""
31
Test that a equals b and they are the same type.
32
Also recursively checks types for list/tuple elements.
33
34
Parameters:
35
- a: First value to compare
36
- b: Second value to compare
37
38
Raises:
39
AssertionError: If values differ or types don't match
40
"""
41
42
def test_ne(a, b):
43
"""
44
Test that a is not equal to b.
45
46
Parameters:
47
- a: First value to compare
48
- b: Second value to compare
49
50
Raises:
51
AssertionError: If values are equal
52
"""
53
54
def test_is(a, b):
55
"""
56
Test that a is b (identity check, not equality).
57
58
Parameters:
59
- a: First object
60
- b: Second object
61
62
Raises:
63
AssertionError: If objects are not identical
64
"""
65
66
def test_close(a, b, eps=1e-5):
67
"""
68
Test that a is within eps of b for numerical comparisons.
69
70
Handles:
71
- Scalar numbers
72
- numpy arrays (element-wise comparison)
73
- Iterables (element-wise comparison)
74
75
Parameters:
76
- a: First numerical value
77
- b: Second numerical value
78
- eps: float, tolerance for comparison (default: 1e-5)
79
80
Raises:
81
AssertionError: If values differ by more than eps
82
"""
83
84
def is_close(a, b, eps=1e-5):
85
"""
86
Check if a is within eps of b without raising exception.
87
88
Parameters:
89
- a: First numerical value
90
- b: Second numerical value
91
- eps: float, tolerance for comparison
92
93
Returns:
94
bool: True if values are within tolerance
95
"""
96
```
97
98
### Exception Testing
99
100
Utilities for testing that code raises expected exceptions with proper error messages.
101
102
```python { .api }
103
def test_fail(f, msg='', contains='', exc=Exception, args=None, kwargs=None):
104
"""
105
Test that function f raises specified exception with optional message checking.
106
107
Parameters:
108
- f: Function to test
109
- msg: str, additional message for assertion failure
110
- contains: str, substring that should be in exception message
111
- exc: Exception class expected to be raised
112
- args: list, arguments to pass to f
113
- kwargs: dict, keyword arguments to pass to f
114
115
Raises:
116
AssertionError: If wrong exception raised or none at all
117
"""
118
119
class ExceptionExpected:
120
"""
121
Context manager for testing expected exceptions with regex matching.
122
123
Parameters:
124
- ex: Exception class to expect (default: Exception)
125
- regex: str, regex pattern to match in exception message
126
127
Usage:
128
with ExceptionExpected(ValueError, "invalid.*format"):
129
risky_operation()
130
"""
131
132
def __init__(self, ex=Exception, regex=''): ...
133
def __enter__(self): ...
134
def __exit__(self, type, value, traceback): ...
135
136
# Convenience instance for common use
137
exception = ExceptionExpected()
138
```
139
140
### Specialized Testing
141
142
Testing functions for specific scenarios like output checking, warnings, and matplotlib figures.
143
144
```python { .api }
145
def test_stdout(f, exp, regex=False):
146
"""
147
Test that function f prints expected output to stdout.
148
149
Parameters:
150
- f: Function to execute
151
- exp: str, expected output (without newline)
152
- regex: bool, treat exp as regex pattern if True
153
154
Raises:
155
AssertionError: If output doesn't match expectation
156
"""
157
158
def test_warns(f, show=False):
159
"""
160
Test that function f produces warnings.
161
162
Parameters:
163
- f: Function to execute
164
- show: bool, print warning details if True
165
166
Raises:
167
AssertionError: If no warnings are raised
168
"""
169
170
def test_fig_exists(ax):
171
"""
172
Test that a matplotlib figure exists and has content.
173
174
Parameters:
175
- ax: matplotlib axes object
176
177
Raises:
178
AssertionError: If no figure or empty figure
179
"""
180
181
def test_shuffled(a, b):
182
"""
183
Test that two sequences contain the same elements but in different order.
184
185
Parameters:
186
- a: First sequence
187
- b: Second sequence
188
189
Raises:
190
AssertionError: If sequences are identical or contain different elements
191
"""
192
```
193
194
### Utility Functions
195
196
Core comparison and testing utilities used by the testing framework.
197
198
```python { .api }
199
def test(a, b, cmp, cname=None):
200
"""
201
Generic test function using custom comparison function.
202
203
Parameters:
204
- a: First value
205
- b: Second value
206
- cmp: Comparison function that returns bool
207
- cname: str, name of comparison for error messages
208
209
Raises:
210
AssertionError: If cmp(a, b) returns False
211
"""
212
213
def nequals(a, b):
214
"""
215
Check if two values are not equal using fastcore's equality checking.
216
217
Parameters:
218
- a: First value
219
- b: Second value
220
221
Returns:
222
bool: True if values are not equal
223
"""
224
```
225
226
### Test Resources
227
228
Pre-defined test resources for common testing scenarios.
229
230
```python { .api }
231
TEST_IMAGE = 'images/puppy.jpg'
232
"""str: Path to test image file for image processing tests."""
233
234
TEST_IMAGE_BW = 'images/mnist3.png'
235
"""str: Path to black and white test image for grayscale processing tests."""
236
```
237
238
## Usage Examples
239
240
### Basic Equality Testing
241
242
```python
243
from fastcore.test import test_eq, test_ne, test_eq_type
244
245
# Basic equality testing
246
test_eq([1, 2, 3], [1, 2, 3]) # Passes
247
test_eq("hello", "hello") # Passes
248
249
# Type checking
250
test_eq_type([1, 2], [1, 2]) # Passes - same values and types
251
test_eq_type([1, 2], (1, 2)) # Fails - different types
252
253
# Inequality testing
254
test_ne([1, 2], [1, 3]) # Passes
255
test_ne("hello", "world") # Passes
256
```
257
258
### Numerical Testing
259
260
```python
261
from fastcore.test import test_close, is_close
262
import numpy as np
263
264
# Floating point comparisons
265
test_close(3.14159, 3.14160, eps=1e-4) # Passes
266
test_close(1.0, 1.0001, eps=1e-3) # Passes
267
268
# Array comparisons
269
arr1 = np.array([1.0, 2.0, 3.0])
270
arr2 = np.array([1.0001, 2.0001, 3.0001])
271
test_close(arr1, arr2, eps=1e-3) # Passes
272
273
# Check without raising exception
274
close = is_close(3.14, 3.15, eps=0.1) # Returns True
275
```
276
277
### Exception Testing
278
279
```python
280
from fastcore.test import test_fail, ExceptionExpected, exception
281
282
# Test specific exception type
283
test_fail(lambda: 1/0, exc=ZeroDivisionError)
284
285
# Test exception with message content
286
test_fail(lambda: int("abc"),
287
contains="invalid literal",
288
exc=ValueError)
289
290
# Using context manager
291
with ExceptionExpected(ValueError, "invalid.*literal"):
292
int("not_a_number")
293
294
# Using convenience instance
295
with exception: # Expects any Exception
296
raise RuntimeError("Something went wrong")
297
```
298
299
### Output and Warning Testing
300
301
```python
302
from fastcore.test import test_stdout, test_warns
303
304
# Test printed output
305
def hello_world():
306
print("Hello, World!")
307
308
test_stdout(hello_world, "Hello, World!")
309
310
# Test with regex
311
def print_number():
312
print("Number: 42")
313
314
test_stdout(print_number, r"Number: \d+", regex=True)
315
316
# Test warnings
317
def deprecated_function():
318
import warnings
319
warnings.warn("This function is deprecated", DeprecationWarning)
320
321
test_warns(deprecated_function) # Passes if warning is raised
322
```
323
324
### Complex Data Testing
325
326
```python
327
from fastcore.test import test_eq, test_shuffled
328
import numpy as np
329
import pandas as pd
330
331
# Testing pandas DataFrames
332
df1 = pd.DataFrame({'A': [1, 2], 'B': [3, 4]})
333
df2 = pd.DataFrame({'A': [1, 2], 'B': [3, 4]})
334
test_eq(df1, df2) # Uses DataFrame-specific equality
335
336
# Testing numpy arrays
337
arr1 = np.array([[1, 2], [3, 4]])
338
arr2 = np.array([[1, 2], [3, 4]])
339
test_eq(arr1, arr2) # Uses array-specific equality
340
341
# Test shuffled sequences
342
original = [1, 2, 3, 4, 5]
343
shuffled = [3, 1, 5, 2, 4]
344
test_shuffled(original, shuffled) # Passes - same elements, different order
345
```
346
347
### Matplotlib Testing
348
349
```python
350
from fastcore.test import test_fig_exists
351
import matplotlib.pyplot as plt
352
353
# Create a plot
354
fig, ax = plt.subplots()
355
ax.plot([1, 2, 3], [1, 4, 9])
356
357
# Test that figure has content
358
test_fig_exists(ax) # Passes - figure has plot data
359
```
360
361
### Custom Testing Scenarios
362
363
```python
364
from fastcore.test import test
365
366
# Custom comparison function
367
def approximately_equal(a, b):
368
return abs(a - b) < 0.1
369
370
# Use custom test
371
test(3.14, 3.15, approximately_equal, "approximately equal")
372
373
# Testing custom objects
374
class Person:
375
def __init__(self, name, age):
376
self.name = name
377
self.age = age
378
def __eq__(self, other):
379
return self.name == other.name and self.age == other.age
380
381
person1 = Person("Alice", 30)
382
person2 = Person("Alice", 30)
383
test_eq(person1, person2) # Uses custom __eq__ method
384
```