0
# Advanced Benchmarking
1

2
Precise measurement control with setup/teardown functions, multiple rounds, and custom iterations for complex benchmarking scenarios. The pedantic API provides fine-grained control over benchmark execution.
3

4
## Capabilities
5

6
### Pedantic Benchmarking
7

8
Advanced benchmarking interface that provides complete control over setup, execution, and teardown phases with configurable rounds and iterations.
9

10
```python { .api }
11
def benchmark.pedantic(
12
    target: Callable[..., T],
13
    args: tuple = (),
14
    kwargs: dict = {},
15
    setup: Callable | None = None,
16
    teardown: Callable | None = None,
17
    rounds: int = 1,
18
    warmup_rounds: int = 0,
19
    iterations: int = 1
20
) -> T:
21
    """
22
    Advanced benchmarking with precise control over execution phases.
23
    
24
    Parameters:
25
    - target: Function to benchmark
26
    - args: Arguments tuple to pass to target function
27
    - kwargs: Keyword arguments dict to pass to target function  
28
    - setup: Function called before each round to prepare state
29
    - teardown: Function called after each round to cleanup state
30
    - rounds: Number of measurement rounds to perform
31
    - warmup_rounds: Number of warmup rounds before measurement
32
    - iterations: Number of target calls per round
33
    
34
    Returns:
35
    The return value of the final target function call
36
    
37
    Raises:
38
    RuntimeError: If benchmark fixture already used in test
39
    ValueError: If invalid parameter combinations provided
40
    """
41
```
42

43
#### Basic Pedantic Usage
44

45
```python
46
def test_with_setup_teardown(benchmark):
47
    data = []
48
    
49
    def setup():
50
        data.clear()
51
        data.extend(range(1000))
52
    
53
    def teardown():
54
        data.clear()
55
        
56
    def sort_data():
57
        return sorted(data)
58
    
59
    result = benchmark.pedantic(
60
        sort_data,
61
        setup=setup,
62
        teardown=teardown,
63
        rounds=10,
64
        warmup_rounds=2
65
    )
66
    assert len(result) == 1000
67
```
68

69
#### Setup Return Values
70

71
When setup returns a value, it replaces the args/kwargs for the target function:
72

73
```python
74
def test_setup_returns_args(benchmark):
75
    def setup():
76
        # Setup returns arguments for the target function
77
        import random
78
        return [random.randint(1, 100) for _ in range(1000)]
79
    
80
    def find_max(numbers):
81
        return max(numbers)
82
    
83
    # setup() return value becomes argument to find_max
84
    result = benchmark.pedantic(
85
        find_max,
86
        setup=setup,
87
        rounds=5
88
    )
89
    assert isinstance(result, int)
90
```
91

92
#### Multiple Iterations
93

94
```python
95
def test_multiple_iterations(benchmark):
96
    counter = {'value': 0}
97
    
98
    def increment():
99
        counter['value'] += 1
100
        return counter['value']
101
    
102
    def reset():
103
        counter['value'] = 0
104
    
105
    # Each round calls increment() 5 times  
106
    result = benchmark.pedantic(
107
        increment,
108
        setup=reset,
109
        rounds=3,
110
        iterations=5
111
    )
112
    assert result == 5  # Last iteration result
113
```
114

115
### Advanced Configuration Options
116

117
Complex parameter combinations and validation rules for pedantic benchmarking.
118

119
```python { .api }
120
# Parameter validation rules:
121
# - rounds must be >= 1
122
# - warmup_rounds must be >= 0  
123
# - iterations must be >= 1
124
# - setup cannot be used with iterations > 1
125
# - setup cannot return value when args/kwargs provided
126
```
127

128
#### Parameter Validation Examples
129

130
```python
131
# Valid configurations
132
benchmark.pedantic(target, rounds=5)                    # Basic multiple rounds
133
benchmark.pedantic(target, warmup_rounds=3, rounds=5)   # With warmup
134
benchmark.pedantic(target, iterations=10)               # Multiple iterations per round
135
benchmark.pedantic(target, setup=setup_func, rounds=5)  # Setup with multiple rounds
136

137
# Invalid configurations - will raise ValueError
138
benchmark.pedantic(target, rounds=0)                    # rounds must be >= 1
139
benchmark.pedantic(target, warmup_rounds=-1)            # warmup_rounds must be >= 0
140
benchmark.pedantic(target, iterations=0)                # iterations must be >= 1
141
benchmark.pedantic(target, setup=setup_func, iterations=5)  # setup conflicts with iterations > 1
142
benchmark.pedantic(target, args=(1,), setup=lambda: (2,))   # setup return conflicts with args
143
```
144

145
### Measurement Mode Differences
146

147
The pedantic API behaves differently depending on the active measurement mode.
148

149
#### Walltime Mode Behavior
150

151
```python
152
# In walltime mode, all parameters are respected
153
result = benchmark.pedantic(
154
    target_function,
155
    rounds=10,          # 10 measurement rounds
156
    iterations=5,       # 5 calls per round
157
    warmup_rounds=3     # 3 warmup rounds before measurement
158
)
159
# Total function calls: 3 * 5 (warmup) + 10 * 5 (measurement) = 65 calls
160
```
161

162
#### Instrumentation Mode Behavior
163

164
```python
165
# In instrumentation mode, rounds/iterations are ignored with warning
166
result = benchmark.pedantic(
167
    target_function,
168
    rounds=10,          # Ignored - warning issued
169
    iterations=5,       # Ignored - warning issued  
170
    warmup_rounds=3     # May be respected for perf trampoline warmup
171
)
172
# Actual behavior: Single measurement call after optional warmup
173
```
174

175
### Error Handling and Constraints
176

177
Common error conditions and their handling in pedantic benchmarking.
178

179
```python
180
# Multiple benchmark calls in one test - RuntimeError
181
def test_invalid_multiple_calls(benchmark):
182
    benchmark(some_function)           # First call OK
183
    benchmark.pedantic(other_function) # RuntimeError: already used
184

185
# Invalid parameter combinations - ValueError  
186
def test_invalid_setup_with_iterations(benchmark):
187
    benchmark.pedantic(
188
        target,
189
        setup=setup_func,
190
        iterations=5  # ValueError: setup cannot be used with iterations > 1
191
    )
192

193
# Setup return value conflicts with provided args - ValueError
194
def test_setup_return_with_args(benchmark):
195
    benchmark.pedantic(
196
        target,
197
        args=(1, 2),        # Providing args
198
        setup=lambda: (3,)  # ValueError: setup cannot return value when args provided
199
    )
200
```
201

202
### Integration with Configuration
203

204
Pedantic options interact with global configuration and marker options.
205

206
```python
207
# Marker options still apply to pedantic calls
208
@pytest.mark.benchmark(group="advanced", max_time=30.0)
209
def test_pedantic_with_marker_options(benchmark):
210
    # Pedantic settings + marker max_time limit
211
    result = benchmark.pedantic(
212
        expensive_function,
213
        rounds=100,  # May be limited by max_time=30.0
214
        iterations=10
215
    )
216
```
217

218
## Best Practices
219

220
### When to Use Pedantic
221

222
- **Complex initialization**: Functions requiring expensive setup before each measurement
223
- **Stateful operations**: Functions that modify global state requiring cleanup
224
- **Multiple rounds**: When you need statistical measurement with custom round counts
225
- **Resource management**: Operations involving files, network connections, or other resources
226

227
### When to Use Basic Benchmark
228

229
- **Pure functions**: Functions with no side effects or complex setup requirements
230
- **Simple measurements**: Single execution time measurement is sufficient  
231
- **Quick iteration**: Rapid development and testing of benchmark scenarios