0
# pytest-xdist
1

2
A pytest plugin that extends pytest with new test execution modes for distributed testing, most importantly across multiple CPUs. It enables parallel test execution through various distribution strategies while maintaining compatibility with pytest's existing ecosystem and other plugins.
3

4
## Package Information
5

6
- **Package Name**: pytest-xdist
7
- **Language**: Python
8
- **Installation**: `pip install pytest-xdist`
9
- **Requires**: pytest >= 7.0.0, execnet >= 2.1
10
- **Optional Dependencies**: psutil (for CPU detection), setproctitle (for process naming)
11

12
## Core Imports
13

14
```python
15
import xdist
16
```
17

18
For using the main API functions:
19

20
```python
21
from xdist import get_xdist_worker_id, is_xdist_worker, is_xdist_controller
22
```
23

24
## Basic Usage
25

26
```python
27
# Command line usage - most common way to use pytest-xdist
28
# Run tests on auto-detected number of CPUs
29
pytest -n auto
30

31
# Run tests on specific number of processes
32
pytest -n 4
33

34
# Run tests with load balancing distribution
35
pytest --dist load
36

37
# Check if running in xdist worker in test code
38
from xdist import is_xdist_worker
39

40
def test_something(request):
41
    if is_xdist_worker(request):
42
        # Code that only runs in worker processes
43
        worker_id = get_xdist_worker_id(request)
44
        print(f"Running in worker {worker_id}")
45
    else:
46
        # Code that only runs in controller/master process
47
        print("Running in controller process")
48
```
49

50
## Architecture
51

52
pytest-xdist implements a controller-worker architecture:
53

54
- **Controller Process**: Manages test collection, coordinates workers, and aggregates results
55
- **Worker Processes**: Execute individual tests and report results back to controller
56
- **Schedulers**: Different algorithms for distributing tests across workers (load, loadscope, loadfile, etc.)
57
- **Communication**: execnet-based communication between controller and workers
58
- **Test Distribution**: Various modes for how tests are distributed (each, load, loadscope, loadfile, loadgroup, worksteal)
59

60
## Capabilities
61

62
### Plugin Installation and Configuration
63

64
Core plugin setup, command-line options, and pytest integration. This includes all the command-line flags and configuration options that control how pytest-xdist operates.
65

66
```python { .api }
67
def pytest_addoption(parser: pytest.Parser) -> None: ...
68
def pytest_configure(config: pytest.Config) -> None: ...
69
def pytest_cmdline_main(config: pytest.Config) -> None: ...
70
```
71

72
[Plugin Configuration](./plugin-configuration.md)
73

74
### Worker Detection and Control
75

76
Functions to detect and control the execution environment, allowing tests and plugins to behave differently when running in distributed mode versus single-process mode.
77

78
```python { .api }
79
def is_xdist_worker(request_or_session) -> bool: ...
80
def is_xdist_controller(request_or_session) -> bool: ...
81
def is_xdist_master(request_or_session) -> bool: ...  # deprecated alias
82
def get_xdist_worker_id(request_or_session) -> str: ...
83
```
84

85
[Worker Detection](./worker-detection.md)
86

87
### Test Distribution and Scheduling
88

89
Multiple scheduling algorithms for distributing tests across workers, each optimized for different test suite characteristics and performance requirements.
90

91
```python { .api }
92
class Scheduling(Protocol):
93
    def add_node(self, node: WorkerController) -> None: ...
94
    def schedule(self) -> None: ...
95
    def mark_test_complete(self, node: WorkerController, item_index: int, duration: float = 0) -> None: ...
96
```
97

98
[Distribution Scheduling](./distribution-scheduling.md)
99

100
### Distributed Session Management
101

102
Core session management for coordinating distributed test execution, including worker lifecycle management and result aggregation.
103

104
```python { .api }
105
class DSession:
106
    def __init__(self, config: pytest.Config) -> None: ...
107
    def session_finished(self) -> bool: ...
108
    def pytest_sessionstart(self, session: pytest.Session) -> None: ...
109
```
110

111
[Session Management](./session-management.md)
112

113
### Hook Specifications
114

115
Custom pytest hooks specific to xdist for plugin authors to extend and customize distributed testing behavior.
116

117
```python { .api }
118
def pytest_xdist_setupnodes(config: pytest.Config, specs: Sequence[execnet.XSpec]) -> None: ...
119
def pytest_configure_node(node: WorkerController) -> None: ...
120
def pytest_testnodeready(node: WorkerController) -> None: ...
121
def pytest_xdist_auto_num_workers(config: pytest.Config) -> int: ...
122
```
123

124
[Hook Specifications](./hook-specifications.md)
125

126
### Loop-on-Fail (Deprecated)
127

128
Legacy functionality for automatically re-running failed tests when files change. This feature is deprecated and will be removed in pytest-xdist 4.0.
129

130
```python { .api }
131
def pytest_cmdline_main(config: pytest.Config) -> int | None: ...
132
def looponfail_main(config: pytest.Config) -> None: ...
133
```
134

135
[Loop-on-Fail](./loop-on-fail.md)
136

137
## Fixtures
138

139
```python { .api }
140
@pytest.fixture(scope="session")
141
def worker_id(request: pytest.FixtureRequest) -> str:
142
    """Return the id of the current worker ('gw0', 'gw1', etc) or 'master'
143
    if running on the master node."""
144

145
@pytest.fixture(scope="session") 
146
def testrun_uid(request: pytest.FixtureRequest) -> str:
147
    """Return the unique id of the current test run."""
148
```
149

150
## Markers
151

152
```python { .api }
153
# Mark tests to run in the same worker session
154
@pytest.mark.xdist_group(name="group_name")
155
def test_example():
156
    pass
157
```
158

159
## Types
160

161
```python { .api }
162
# Main protocol for scheduling implementations
163
class Scheduling(Protocol):
164
    @property
165
    def nodes(self) -> list[WorkerController]: ...
166
    @property 
167
    def collection_is_completed(self) -> bool: ...
168
    @property
169
    def tests_finished(self) -> bool: ...
170
    @property
171
    def has_pending(self) -> bool: ...
172
    
173
    def add_node(self, node: WorkerController) -> None: ...
174
    def add_node_collection(self, node: WorkerController, collection: Sequence[str]) -> None: ...
175
    def mark_test_complete(self, node: WorkerController, item_index: int, duration: float = 0) -> None: ...
176
    def mark_test_pending(self, item: str) -> None: ...
177
    def remove_pending_tests_from_node(self, node: WorkerController, indices: Sequence[int]) -> None: ...
178
    def remove_node(self, node: WorkerController) -> str | None: ...
179
    def schedule(self) -> None: ...
180

181
# Worker controller for managing individual worker processes
182
class WorkerController: ...
183

184
# Distributed session class for coordinating execution
185
class DSession: ...
186

187
# Node manager for setting up and managing workers
188
class NodeManager: ...
189

190
# Remote execution utilities
191
class Producer: ...
192
class TestQueue: ...
193
```