0
# pytest-benchmark
1
2
## Overview
3
4
pytest-benchmark is a pytest plugin that provides a comprehensive fixture for benchmarking Python code. It automatically calibrates test runs to provide accurate performance measurements, integrates seamlessly with pytest's testing framework, and offers statistical analysis of results with multiple output formats.
5
6
## Package Information
7
8
- **Name**: pytest-benchmark
9
- **Type**: pytest plugin / Python library
10
- **Language**: Python
11
- **Installation**: `pip install pytest-benchmark`
12
13
## Core Imports
14
15
```python
16
# Primary usage - the benchmark fixture is automatically available in pytest tests
17
import pytest
18
19
# For programmatic access to benchmarking classes (rarely needed)
20
from pytest_benchmark.fixture import BenchmarkFixture
21
from pytest_benchmark.session import BenchmarkSession
22
```
23
24
## Basic Usage
25
26
### Simple Function Benchmarking
27
28
```python
29
def test_my_function(benchmark):
30
# Benchmark a function with automatic calibration
31
result = benchmark(my_function, arg1, arg2, kwarg=value)
32
assert result == expected_value
33
```
34
35
```python
36
def my_function(x, y):
37
"""Function to be benchmarked."""
38
return x * y + sum(range(100))
39
40
def test_benchmark_example(benchmark):
41
result = benchmark(my_function, 5, 10)
42
assert result == 4999
43
```
44
45
### Pedantic Mode for Precise Control
46
47
```python
48
def test_pedantic_benchmark(benchmark):
49
# Fine-grained control over benchmark execution
50
result = benchmark.pedantic(
51
target=my_function,
52
args=(5, 10),
53
rounds=10,
54
iterations=1000
55
)
56
assert result == 4999
57
```
58
59
## Capabilities
60
61
### Core Benchmarking
62
63
The main benchmarking functionality through the `benchmark` fixture that provides automatic calibration, warmup, and statistical analysis.
64
65
**Key APIs:**
66
67
```python
68
def benchmark(func, *args, **kwargs) -> Any: ...
69
def benchmark.pedantic(target, args=(), kwargs=None, setup=None, rounds=1, warmup_rounds=0, iterations=1) -> Any: ...
70
```
71
72
[Core Benchmarking](./core-benchmarking.md)
73
74
### Aspect-Oriented Benchmarking
75
76
Benchmark existing functions without modifying test code using aspect-oriented programming via the `weave` method.
77
78
**Key APIs:**
79
80
```python
81
def benchmark.weave(target, **kwargs) -> None: ...
82
benchmark_weave = benchmark.weave # Shortcut fixture
83
```
84
85
[Aspect-Oriented Benchmarking](./aspect-benchmarking.md)
86
87
### Configuration and Customization
88
89
Extensive configuration options via pytest command-line arguments and test markers for controlling benchmark behavior.
90
91
**Key APIs:**
92
93
```python
94
@pytest.mark.benchmark(max_time=2.0, min_rounds=10, group="mygroup")
95
def test_example(benchmark): ...
96
```
97
98
[Configuration and Customization](./configuration.md)
99
100
### Results Storage and Comparison
101
102
Store benchmark results in various backends (file, Elasticsearch) and compare performance across runs.
103
104
**Key APIs:**
105
106
```bash
107
pytest --benchmark-save=baseline
108
pytest --benchmark-compare
109
pytest-benchmark compare
110
```
111
112
[Results Storage and Comparison](./storage-comparison.md)
113
114
### Statistical Analysis and Reporting
115
116
Comprehensive statistical analysis with multiple output formats including tables, CSV, histograms, and cProfile integration.
117
118
**Key APIs:**
119
120
```bash
121
pytest --benchmark-histogram --benchmark-csv=results.csv
122
pytest --benchmark-cprofile=cumtime
123
```
124
125
[Statistical Analysis and Reporting](./analysis-reporting.md)
126
127
### Command Line Interface
128
129
Standalone CLI tools for managing and analyzing saved benchmark results outside of pytest.
130
131
**Key APIs:**
132
133
```bash
134
pytest-benchmark list
135
pytest-benchmark compare [options] [glob_or_file...]
136
pytest-benchmark help [command]
137
```
138
139
[Command Line Interface](./cli-tools.md)