0
# pytest-randomly
1

2
A pytest plugin that introduces controlled randomness into test execution to improve test quality and discover hidden dependencies. It randomly shuffles test execution order and resets random seeds to ensure reproducible test runs.
3

4
## Package Information
5

6
- **Package Name**: pytest-randomly
7
- **Package Type**: pypi
8
- **Language**: Python
9
- **Installation**: `pip install pytest-randomly`
10

11
## Core Imports
12

13
```python
14
import pytest_randomly
15
```
16

17
The plugin is automatically discovered by pytest via entry points - no explicit imports needed.
18

19
## Basic Usage
20

21
```python
22
# Install the plugin
23
# pip install pytest-randomly
24

25
# Run tests with automatic randomization (default behavior)
26
pytest
27

28
# Run tests with specific seed for reproducibility
29
pytest --randomly-seed=1234
30

31
# Reuse seed from previous run
32
pytest --randomly-seed=last
33

34
# Disable test order shuffling but keep seed resetting
35
pytest --randomly-dont-reorganize
36

37
# Disable seed resetting but keep order shuffling
38
pytest --randomly-dont-reset-seed
39

40
# Completely disable the plugin
41
pytest -p no:randomly
42
```
43

44
## Architecture
45

46
The plugin provides controlled randomness through:
47

48
- **Test Order Randomization**: Shuffles tests at module, class, and function levels using deterministic seeding
49
- **Random State Management**: Resets global random seeds before each test for reproducibility
50
- **Third-party Integration**: Automatically resets random states for factory-boy, faker, model-bakery, and numpy
51
- **pytest-xdist Support**: Distributes consistent seeds across parallel worker processes
52
- **Extensibility**: Entry point system for third-party random seeder registration
53

54
## Capabilities
55

56
### Plugin Configuration
57

58
Adds command-line options and configures the plugin when pytest starts.
59

60
```python { .api }
61
def pytest_addoption(parser: Parser) -> None:
62
    """
63
    Adds command-line options for controlling randomness behavior.
64
    
65
    Options added:
66
    - --randomly-seed: Set seed value (int, 'default', or 'last')
67
    - --randomly-dont-reset-seed: Disable random seed reset per test
68
    - --randomly-dont-reorganize: Disable test order shuffling
69
    """
70

71
def pytest_configure(config: Config) -> None:
72
    """
73
    Configures the plugin when pytest starts.
74
    
75
    Handles seed generation, xdist integration, and caching.
76
    """
77
```
78

79
### Test Execution Hooks
80

81
Controls random state during different phases of test execution.
82

83
```python { .api }
84
def pytest_runtest_setup(item: Item) -> None:
85
    """
86
    Resets random state before each test setup phase.
87
    
88
    Args:
89
        item: The test item being set up
90
    """
91

92
def pytest_runtest_call(item: Item) -> None:
93
    """
94
    Resets random state before each test execution phase.
95
    
96
    Args:
97
        item: The test item being executed
98
    """
99

100
def pytest_runtest_teardown(item: Item) -> None:
101
    """
102
    Resets random state after each test teardown phase.
103
    
104
    Args:
105
        item: The test item being torn down
106
    """
107
```
108

109
### Test Collection and Ordering
110

111
Randomly reorders collected test items for better test isolation detection.
112

113
```python { .api }
114
@hookimpl(tryfirst=True)
115
def pytest_collection_modifyitems(config: Config, items: list[Item]) -> None:
116
    """
117
    Randomly reorders collected test items by module, class, then function.
118
    
119
    Uses deterministic shuffling based on seed value to ensure
120
    reproducible test order when using the same seed.
121
    
122
    Args:
123
        config: pytest configuration object
124
        items: List of collected test items to reorder
125
    """
126
```
127

128
### Reporting
129

130
Provides information about the randomization seed being used.
131

132
```python { .api }
133
def pytest_report_header(config: Config) -> str:
134
    """
135
    Returns header line showing the seed being used.
136
    
137
    Args:
138
        config: pytest configuration object
139
        
140
    Returns:
141
        Header string with seed information (e.g., "Using --randomly-seed=1234")
142
    """
143
```
144

145
### Utility Functions
146

147
Helper functions for seed validation and argument parsing.
148

149
```python { .api }
150
def seed_type(string: str) -> str | int:
151
    """
152
    Argument parser type converter for --randomly-seed option.
153
    
154
    Args:
155
        string: String value to convert
156
        
157
    Returns:
158
        Integer seed value, or 'default'/'last' string values
159
        
160
    Raises:
161
        argparse.ArgumentTypeError: For invalid seed values
162
    """
163

164
def reduce_list_of_lists(lists: list[list[T]]) -> list[T]:
165
    """
166
    Flattens a list of lists into a single list.
167
    
168
    Args:
169
        lists: List of lists to flatten
170
        
171
    Returns:
172
        Single flattened list containing all elements
173
    """
174
```
175

176
### pytest-xdist Integration
177

178
Handles parallel test execution with consistent seeding across workers.
179

180
```python { .api }
181
class XdistHooks:
182
    """
183
    Hooks container for pytest-xdist integration.
184
    
185
    Automatically registered when xdist plugin is detected.
186
    """
187
    
188
    def pytest_configure_node(self, node: Item) -> None:
189
        """
190
        Configures worker nodes in xdist with consistent seed values.
191
        
192
        Args:
193
            node: Worker node to configure
194
        """
195
```
196

197
### faker Integration
198

199
Provides seed configuration for the faker pytest plugin when faker is installed.
200

201
**Note**: This fixture is only defined when the `faker` package is available at import time.
202

203
```python { .api }
204
# Only available when faker is installed
205
if have_faker:
206
    @fixture(autouse=True)
207
    def faker_seed(pytestconfig: Config) -> int:
208
        """
209
        Provides faker seed configuration to faker pytest plugin.
210
        
211
        Conditionally defined fixture that only exists when faker package 
212
        is available at plugin import time.
213
        
214
        Args:
215
            pytestconfig: pytest configuration object
216
            
217
        Returns:
218
            Current randomly seed value for faker integration
219
        """
220
```
221

222
## Types
223

224
```python { .api }
225
# Module-level variables
226
default_seed: int  # Default random seed generated at import time
227
random_states: dict[int, tuple[Any, ...]]  # Cache for random states by seed
228
np_random_states: dict[int, Any]  # Cache for numpy random states by seed
229
entrypoint_reseeds: list[Callable[[int], None]] | None  # Cached entry point reseeder functions
230

231
# Third-party library availability flags
232
have_factory_boy: bool  # True if factory-boy is available
233
have_faker: bool        # True if faker is available
234
have_model_bakery: bool # True if model-bakery is available
235
have_numpy: bool        # True if numpy is available
236

237
# Optional module imports
238
xdist: ModuleType | None  # pytest-xdist module if available, None otherwise
239

240
# Type variable for generic list flattening
241
T = TypeVar("T")
242
```
243

244
## Third-party Integration
245

246
The plugin automatically detects and integrates with popular Python testing libraries when they are available:
247

248
- **factory-boy** (if `have_factory_boy` is True): Resets `factory.random` state for consistent factory generation
249
- **faker** (if `have_faker` is True): Resets `faker.generator.random` state for reproducible fake data
250
- **model-bakery** (if `have_model_bakery` is True): Resets `model_bakery.random_gen.baker_random` state
251
- **numpy** (if `have_numpy` is True): Resets `numpy.random` legacy global state for consistent array generation
252
- **pytest-xdist** (if detected): Distributes seed values to parallel worker processes
253

254
## Extension API
255

256
Third-party packages can register custom random seeders via the `pytest_randomly.random_seeder` entry point group:
257

258
```python
259
# In your setup.py or pyproject.toml
260
entry_points = {
261
    "pytest_randomly.random_seeder": [
262
        "your_lib = your_package:reseed_function"
263
    ]
264
}
265

266
# Your reseed function signature
267
def reseed_function(seed: int) -> None:
268
    """Reset your random generator with the provided seed."""
269
    your_random_generator.seed(seed)
270
```
271

272
## Command Line Options
273

274
- `--randomly-seed=<int|'default'|'last'>`: Control the random seed value
275
- `--randomly-dont-reset-seed`: Disable per-test seed reset while keeping order shuffling
276
- `--randomly-dont-reorganize`: Disable test order shuffling while keeping seed reset
277
- `-p no:randomly`: Completely disable the plugin