0
# pytest-rerunfailures
1

2
A pytest plugin that automatically re-runs failed tests to eliminate intermittent failures and flaky test behavior. It provides comprehensive command-line options for controlling test reruns including global rerun counts, selective rerun filtering based on exception patterns, configurable delays between reruns, and marker-based test decoration for individual test control.
3

4
## Package Information
5

6
- **Package Name**: pytest-rerunfailures
7
- **Package Type**: pytest plugin
8
- **Language**: Python
9
- **Installation**: `pip install pytest-rerunfailures`
10
- **Requirements**: Python >=3.9, pytest >=7.4 (!=8.2.2)
11
- **Optional**: pytest-xdist >=2.3.0 for crash recovery
12

13
## Core Imports
14

15
The plugin is automatically discovered by pytest when installed. No explicit imports are needed for basic usage.
16

17
For accessing plugin internals programmatically:
18

19
```python
20
import pytest_rerunfailures
21
```
22

23
## Basic Usage
24

25
### Command Line Usage
26

27
Re-run all test failures up to 5 times:
28

29
```bash
30
pytest --reruns 5
31
```
32

33
Add 1-second delay between reruns:
34

35
```bash
36
pytest --reruns 5 --reruns-delay 1
37
```
38

39
Only rerun specific error types:
40

41
```bash
42
pytest --reruns 3 --only-rerun AssertionError --only-rerun ValueError
43
```
44

45
Skip reruns for specific error types:
46

47
```bash
48
pytest --reruns 3 --rerun-except ConnectionError
49
```
50

51
### Marker Usage
52

53
Mark individual tests for reruns:
54

55
```python
56
import pytest
57

58
@pytest.mark.flaky(reruns=5)
59
def test_sometimes_fails():
60
    import random
61
    assert random.choice([True, False])
62

63
@pytest.mark.flaky(reruns=3, reruns_delay=2)
64
def test_with_delay():
65
    # Will be retried up to 3 times with 2-second delays
66
    pass
67

68
@pytest.mark.flaky(reruns=2, condition="sys.platform.startswith('win32')")
69
def test_windows_only_reruns():
70
    # Only reruns on Windows
71
    pass
72
```
73

74
## Capabilities
75

76
### Command Line Options
77

78
Global configuration options for controlling test reruns across the entire test suite.
79

80
```python { .api }
81
def pytest_addoption(parser):
82
    """
83
    Adds command-line options for test reruns.
84
    
85
    Options added:
86
    --reruns: Number of times to re-run failed tests (default: 0)
87
    --reruns-delay: Delay in seconds between reruns
88
    --only-rerun: Regex patterns for errors to rerun (repeatable)
89
    --rerun-except: Regex patterns for errors NOT to rerun (repeatable)
90
    --fail-on-flaky: Fail with exit code 7 if flaky test passes on rerun
91
    """
92
```
93

94
### Test Execution Control
95

96
Core functions for controlling test execution and rerun behavior.
97

98
```python { .api }
99
def get_reruns_count(item):
100
    """
101
    Get the number of reruns for a test item.
102
    Priority: marker > command line > ini file
103
    
104
    Args:
105
        item: pytest test item
106
        
107
    Returns:
108
        int: Number of reruns for this test
109
    """
110

111
def get_reruns_delay(item):
112
    """
113
    Get the delay between reruns for a test item.
114
    
115
    Args:
116
        item: pytest test item
117
        
118
    Returns:
119
        float: Delay in seconds between reruns
120
    """
121

122
def get_reruns_condition(item):
123
    """
124
    Evaluate whether reruns should happen for a test item.
125
    
126
    Args:
127
        item: pytest test item
128
        
129
    Returns:
130
        bool: Whether reruns are enabled for this test
131
    """
132

133
def evaluate_condition(item, mark, condition: object) -> bool:
134
    """
135
    Evaluate condition expressions for flaky marker.
136
    
137
    Args:
138
        item: pytest test item
139
        mark: pytest marker object
140
        condition: str or bool condition to evaluate
141
        
142
    Returns:
143
        bool: Result of condition evaluation
144
    """
145
```
146

147
### Pytest Hooks
148

149
Plugin hooks that integrate with pytest's test execution lifecycle.
150

151
```python { .api }
152
def pytest_configure(config):
153
    """
154
    Configure the plugin and register flaky marker.
155
    
156
    Args:
157
        config: pytest configuration object
158
    """
159

160
def pytest_runtest_protocol(item, nextitem):
161
    """
162
    Main test execution protocol with rerun logic.
163
    
164
    Args:
165
        item: pytest test item to execute
166
        nextitem: next test item in queue
167
        
168
    Returns:
169
        bool: True if protocol handled (reruns enabled), None for default behavior
170
    """
171

172
def pytest_runtest_teardown(item, nextitem):
173
    """
174
    Handle test teardown with rerun logic.
175
    
176
    Args:
177
        item: pytest test item
178
        nextitem: next test item in queue
179
    """
180

181
@pytest.hookimpl(hookwrapper=True)
182
def pytest_runtest_makereport(item, call):
183
    """
184
    Create test reports with rerun tracking.
185
    
186
    Args:
187
        item: pytest test item
188
        call: pytest call info
189
        
190
    Returns:
191
        TestReport: Enhanced with rerun information
192
    """
193

194
def pytest_report_teststatus(report):
195
    """
196
    Report test status including rerun status.
197
    
198
    Args:
199
        report: pytest test report
200
        
201
    Returns:
202
        tuple: (outcome, letter, verbose_word) for rerun status
203
    """
204

205
def pytest_terminal_summary(terminalreporter):
206
    """
207
    Display rerun summary in terminal output.
208
    
209
    Args:
210
        terminalreporter: pytest terminal reporter
211
    """
212

213
@pytest.hookimpl(trylast=True)
214
def pytest_sessionfinish(session, exitstatus):
215
    """
216
    Handle session finish with flaky test detection.
217
    
218
    Args:
219
        session: pytest session
220
        exitstatus: current exit status
221
    """
222
```
223

224
### Test State Management
225

226
Functions for managing test state during reruns, including fixture cleanup and setup state management.
227

228
```python { .api }
229
def check_options(config):
230
    """
231
    Validate command-line options compatibility.
232
    
233
    Args:
234
        config: pytest configuration object
235
        
236
    Raises:
237
        pytest.UsageError: If incompatible options are used
238
    """
239

240
def works_with_current_xdist():
241
    """
242
    Check compatibility with pytest-xdist version.
243
    
244
    Returns:
245
        bool or None: True if compatible, False if incompatible, None if not installed
246
    """
247

248
def is_master(config):
249
    """
250
    Determine if running on master node in distributed tests.
251
    
252
    Args:
253
        config: pytest configuration object
254
        
255
    Returns:
256
        bool: True if master node, False if worker
257
    """
258

259
def show_rerun(terminalreporter, lines):
260
    """
261
    Display rerun test information.
262
    
263
    Args:
264
        terminalreporter: pytest terminal reporter
265
        lines: list to append rerun information to
266
    """
267
```
268

269
### Distributed Testing Support
270

271
Classes and hooks for pytest-xdist integration and crash recovery.
272

273
```python { .api }
274
class XDistHooks:
275
    """Hooks for pytest-xdist integration."""
276
    
277
    def pytest_configure_node(self, node):
278
        """
279
        Configure xdist worker nodes.
280
        
281
        Args:
282
            node: xdist worker node
283
        """
284
    
285
    def pytest_handlecrashitem(self, crashitem, report, sched):
286
        """
287
        Handle crashed tests in distributed execution.
288
        
289
        Args:
290
            crashitem: crashed test item identifier
291
            report: crash report
292
            sched: xdist scheduler
293
        """
294
```
295

296
### Status Database Classes
297

298
Database classes for tracking test failures and reruns across distributed test execution.
299

300
```python { .api }
301
class StatusDB:
302
    """Base in-memory database for tracking test reruns and failures."""
303
    
304
    def add_test_failure(self, crashitem):
305
        """
306
        Record a test failure.
307
        
308
        Args:
309
            crashitem: test item identifier
310
        """
311
    
312
    def get_test_failures(self, crashitem):
313
        """
314
        Get failure count for a test.
315
        
316
        Args:
317
            crashitem: test item identifier
318
            
319
        Returns:
320
            int: Number of failures recorded
321
        """
322
    
323
    def set_test_reruns(self, crashitem, reruns):
324
        """
325
        Set rerun count for a test.
326
        
327
        Args:
328
            crashitem: test item identifier
329
            reruns: number of reruns allowed
330
        """
331
    
332
    def get_test_reruns(self, crashitem):
333
        """
334
        Get rerun count for a test.
335
        
336
        Args:
337
            crashitem: test item identifier
338
            
339
        Returns:
340
            int: Number of reruns allowed
341
        """
342

343
class ServerStatusDB(StatusDB):
344
    """Server-side socket database for distributed testing."""
345
    
346
    @property
347
    def sock_port(self):
348
        """
349
        Get the socket port for client connections.
350
        
351
        Returns:
352
            int: Port number
353
        """
354
    
355
    def run_server(self):
356
        """Run the database server to handle client connections."""
357

358
class ClientStatusDB(StatusDB):
359
    """Client-side socket database for distributed testing."""
360
    
361
    def __init__(self, sock_port):
362
        """
363
        Initialize client database.
364
        
365
        Args:
366
            sock_port: server socket port to connect to
367
        """
368
```
369

370
## Marker Reference
371

372
### @pytest.mark.flaky
373

374
Mark individual tests for automatic reruns on failure.
375

376
```python { .api }
377
@pytest.mark.flaky(
378
    reruns=1,                    # int: Number of reruns (default: 1)
379
    reruns_delay=0,              # float: Delay between reruns in seconds
380
    condition=True,              # str|bool: Condition for when reruns should happen
381
    only_rerun=None,             # str|list[str]: Regex patterns for errors to rerun
382
    rerun_except=None            # str|list[str]: Regex patterns for errors NOT to rerun
383
)
384
def test_function():
385
    """Test that may be flaky and need reruns."""
386
```
387

388
## Configuration
389

390
### Command Line Options
391

392
- `--reruns N`: Re-run failed tests N times (default: 0)
393
- `--reruns-delay SECONDS`: Add delay between reruns (default: 0)
394
- `--only-rerun REGEX`: Only rerun errors matching regex (repeatable)
395
- `--rerun-except REGEX`: Skip reruns for errors matching regex (repeatable)
396
- `--fail-on-flaky`: Fail with exit code 7 if flaky test passes on rerun
397

398
### Configuration File Options
399

400
In `pytest.ini` or `pyproject.toml`:
401

402
```ini
403
[tool.pytest.ini_options]
404
reruns = "3"
405
reruns_delay = "1"
406
```
407

408
### Priority Order
409

410
1. `@pytest.mark.flaky` marker (highest priority)
411
2. Command-line options (`--reruns`, `--reruns-delay`)
412
3. Configuration file options (lowest priority)
413

414
## Error Handling
415

416
### Compatibility Restrictions
417

418
- Not compatible with `--pdb` flag
419
- Not compatible with `--looponfail` flag (pytest-xdist)
420
- Not compatible with `flaky` plugin (choose one or the other)
421

422
### Exception Filtering
423

424
Control which exceptions trigger reruns:
425

426
```python
427
# Only rerun on specific exceptions
428
@pytest.mark.flaky(only_rerun=["AssertionError", "ConnectionError"])
429
def test_network_operation():
430
    pass
431

432
# Skip reruns for specific exceptions
433
@pytest.mark.flaky(rerun_except="KeyboardInterrupt")
434
def test_user_interaction():
435
    pass
436
```
437

438
### Conditional Reruns
439

440
Use conditions to control when reruns happen:
441

442
```python
443
import sys
444

445
# Only rerun on Windows
446
@pytest.mark.flaky(reruns=3, condition="sys.platform.startswith('win32')")
447
def test_windows_specific():
448
    pass
449

450
# Only rerun in CI environment
451
@pytest.mark.flaky(reruns=2, condition="'CI' in os.environ")
452
def test_ci_sensitive():
453
    pass
454
```
455

456
## Advanced Usage
457

458
### Crash Recovery
459

460
With pytest-xdist installed, the plugin can recover from hard crashes:
461

462
```bash
463
# Enable crash recovery with parallel execution
464
pytest -n auto --reruns 3
465
```
466

467
### Accessing Execution Count
468

469
In custom pytest hooks, access the execution count:
470

471
```python
472
@pytest.hookimpl(tryfirst=True)
473
def pytest_runtest_makereport(item, call):
474
    if hasattr(item, 'execution_count'):
475
        print(f"Test {item.nodeid} executed {item.execution_count} times")
476
```
477

478
### Test Item Attributes
479

480
The plugin adds these attributes to test items:
481

482
- `execution_count`: Number of times the test has been executed
483
- `_test_failed_statuses`: Dict tracking failure status for each test phase
484
- `_terminal_errors`: Dict tracking terminal errors that prevent reruns
485

486
### Report Attributes
487

488
The plugin adds these attributes to test reports:
489

490
- `rerun`: Rerun number for this report (0 for first run)
491
- `outcome`: May be "rerun" for intermediate rerun reports
492

493
## Types
494

495
```python { .api }
496
# Test item attributes added by plugin
497
class TestItem:
498
    execution_count: int
499
    _test_failed_statuses: dict[str, bool]
500
    _terminal_errors: dict[str, bool]
501

502
# Report attributes added by plugin  
503
class TestReport:
504
    rerun: int  # 0 for first run, 1+ for reruns
505
    outcome: str  # may be "rerun" for intermediate reports
506

507
# Configuration constants
508
HAS_PYTEST_HANDLECRASHITEM: bool  # True if xdist crash handling available
509
RERUNS_DESC: str  # Description for --reruns option
510
RERUNS_DELAY_DESC: str  # Description for --reruns-delay option
511
suspended_finalizers: dict  # Global storage for suspended test finalizers
512
```