0
# pytest-subtests
1

2
A pytest plugin that provides unittest's subTest() support and introduces a subtests fixture for pure pytest tests. It enables developers to run multiple related test cases within a single test function while maintaining individual failure reporting and isolation, making it particularly useful for parameterized testing scenarios where you want to see all failures rather than stopping at the first one.
3

4
## Package Information
5

6
- **Package Name**: pytest-subtests
7
- **Package Type**: pypi
8
- **Language**: Python
9
- **Installation**: `pip install pytest-subtests`
10
- **Python Versions**: >=3.9
11
- **Dependencies**: pytest >=7.4, attrs >=19.2.0
12

13
## Core Imports
14

15
```python
16
import pytest_subtests
17
from pytest_subtests import SubTests
18
```
19

20
For use in test functions, the `subtests` fixture is automatically available:
21

22
```python
23
def test_example(subtests):
24
    # subtests fixture is automatically injected
25
    pass
26
```
27

28
The plugin registers automatically when pytest-subtests is installed - no manual registration needed.
29

30
## Basic Usage
31

32
### Using the subtests fixture (pure pytest style)
33

34
```python
35
def test_with_subtests(subtests):
36
    for i in range(5):
37
        with subtests.test(msg="custom message", i=i):
38
            assert i % 2 == 0
39
```
40

41
### Using unittest.TestCase.subTest (unittest compatibility)
42

43
```python
44
import unittest
45

46
class TestExample(unittest.TestCase):
47
    def test_with_subtests(self):
48
        for i in range(5): 
49
            with self.subTest("custom message", i=i):
50
                self.assertEqual(i % 2, 0)
51
```
52

53
Both approaches provide individual failure reporting - if some subtests pass and others fail, you'll see detailed output for each failing subtest while still getting information about which ones passed.
54

55
## Capabilities
56

57
### SubTests Class
58

59
Main interface for creating subtests within pytest functions.
60

61
```python { .api }
62
class SubTests:
63
    """Primary interface for creating subtests within pytest functions."""
64
    
65
    @property
66
    def item(self) -> pytest.Item:
67
        """Returns the current test item."""
68
        
69
    def test(self, msg: str | None = None, **kwargs: Any) -> _SubTestContextManager:
70
        """
71
        Creates a subtest context manager.
72
        
73
        Args:
74
            msg: Optional message for the subtest
75
            **kwargs: Parameters for subtest identification
76
            
77
        Returns:
78
            Context manager for the subtest
79
        """
80
```
81

82
### Subtests Fixture
83

84
Pytest fixture that provides a SubTests instance to test functions.
85

86
```python { .api }
87
def subtests(request: SubRequest) -> Generator[SubTests, None, None]:
88
    """
89
    Pytest fixture providing SubTests instance to test functions.
90
    
91
    Args:
92
        request: pytest sub-request object
93
        
94
    Yields:
95
        SubTests instance for creating subtests
96
    """
97
```
98

99
### SubTestContext
100

101
Container for subtest context information.
102

103
```python { .api }
104
class SubTestContext:
105
    """Container for subtest context information."""
106
    
107
    msg: str | None
108
    """Optional message for the subtest."""
109
    
110
    kwargs: dict[str, Any] 
111
    """Parameters for subtest identification."""
112
```
113

114
### SubTestReport
115

116
Custom test report for subtests with enhanced formatting.
117

118
```python { .api }
119
class SubTestReport(TestReport):
120
    """Custom test report for subtests with enhanced formatting."""
121
    
122
    context: SubTestContext
123
    """Subtest context information."""
124
    
125
    @property
126
    def head_line(self) -> str:
127
        """Returns formatted header line for the report."""
128
        
129
    def sub_test_description(self) -> str:
130
        """
131
        Generates human-readable subtest description.
132
        
133
        Returns:
134
            Human-readable description of the subtest
135
        """
136
        
137
    def _to_json(self) -> dict:
138
        """Serializes the report to JSON format."""
139
        
140
    @classmethod
141
    def _from_json(cls, reportdict: dict[str, Any]) -> SubTestReport:
142
        """Creates SubTestReport from JSON data."""
143
        
144
    @classmethod  
145
    def _from_test_report(cls, test_report: TestReport) -> SubTestReport:
146
        """Creates SubTestReport from a regular TestReport."""
147
```
148

149
### SubTest Context Manager
150

151
Context manager returned by `SubTests.test()` that handles subtest execution and reporting.
152

153
```python { .api }
154
class _SubTestContextManager:
155
    """
156
    Context manager for subtests, capturing exceptions and handling them 
157
    through the pytest machinery.
158
    """
159
    
160
    def __enter__(self) -> None:
161
        """Enters the subtest context, sets up capturing and timing."""
162
        
163
    def __exit__(
164
        self, 
165
        exc_type: type[Exception] | None,
166
        exc_val: Exception | None, 
167
        exc_tb: TracebackType | None
168
    ) -> bool:
169
        """
170
        Exits the subtest context, processes any exceptions and generates reports.
171
        
172
        Returns:
173
            True to suppress the exception (subtest handling), False otherwise
174
        """
175
```
176

177
### Command Line Options
178

179
The plugin adds command line options to control subtest behavior.
180

181
```python { .api }
182
def pytest_addoption(parser: pytest.Parser) -> None:
183
    """
184
    Adds command-line options for subtest behavior.
185
    
186
    Args:
187
        parser: pytest argument parser to add options to
188
    
189
    Adds options:
190
        --no-subtests-shortletter: Disables subtest output 'dots' in 
191
                                  non-verbose mode (EXPERIMENTAL)
192
    """
193
```
194

195
### Pytest Hook Integration
196

197
The plugin integrates with pytest through several hooks for configuration and reporting.
198

199
```python { .api }
200
def pytest_configure(config: pytest.Config) -> None:
201
    """
202
    Configures plugin, patches TestCaseFunction for unittest compatibility.
203
    
204
    Args:
205
        config: pytest configuration object
206
        
207
    Performs:
208
        - Patches TestCaseFunction.addSubTest for subtest support
209
        - Sets failfast=False to allow subtests to continue on failure
210
        - Adds subtest status types to terminal reporter
211
        - Updates color mapping for subtest outcomes
212
    """
213

214
def pytest_unconfigure() -> None:
215
    """
216
    Cleans up plugin modifications when plugin is unconfigured.
217
    
218
    Removes:
219
        - TestCaseFunction.addSubTest attribute
220
        - TestCaseFunction.failfast attribute  
221
        - Restores original TestCaseFunction.addSkip method
222
    """
223

224
def pytest_report_teststatus(
225
    report: pytest.TestReport, 
226
    config: pytest.Config
227
) -> tuple[str, str, str | Mapping[str, bool]] | None:
228
    """
229
    Customizes test status reporting for subtests.
230
    
231
    Args:
232
        report: test report to process
233
        config: pytest configuration
234
        
235
    Returns:
236
        Tuple of (category, shortletter, verbose) for subtest reports,
237
        None for non-subtest reports to let other handlers process them
238
        
239
    Handles:
240
        - SubTestReport instances with custom formatting
241
        - xfail/xpass status for subtests  
242
        - Custom short letters (, - u y Y) for subtest outcomes
243
        - Respects --no-subtests-shortletter option
244
    """
245

246
def pytest_report_to_serializable(
247
    report: pytest.TestReport
248
) -> dict[str, Any] | None:
249
    """
250
    Handles serialization of SubTestReport objects for distributed testing.
251
    
252
    Args:
253
        report: test report to serialize
254
        
255
    Returns:
256
        JSON-serializable dict for SubTestReport, None for other reports
257
    """
258

259
def pytest_report_from_serializable(
260
    data: dict[str, Any]
261
) -> SubTestReport | None:
262
    """
263
    Handles deserialization of SubTestReport objects from distributed testing.
264
    
265
    Args:
266
        data: serialized report data
267
        
268
    Returns:
269
        SubTestReport instance if data represents a SubTestReport, None otherwise
270
    """
271
```
272

273
## Usage Examples
274

275
### Parameterized Testing with Individual Failure Reporting
276

277
```python
278
def test_multiple_values(subtests):
279
    test_data = [
280
        (2, 4),    # pass
281
        (3, 6),    # pass  
282
        (4, 7),    # fail - should be 8
283
        (5, 10),   # pass
284
    ]
285
    
286
    for input_val, expected in test_data:
287
        with subtests.test(input=input_val, expected=expected):
288
            result = input_val * 2
289
            assert result == expected
290
```
291

292
### Data Validation with Contextual Information
293

294
```python
295
def test_user_data_validation(subtests):
296
    users = [
297
        {"name": "Alice", "age": 25, "email": "alice@example.com"},
298
        {"name": "Bob", "age": -5, "email": "invalid-email"},  # multiple issues
299
        {"name": "", "age": 30, "email": "charlie@example.com"},  # empty name
300
    ]
301
    
302
    for i, user in enumerate(users):
303
        with subtests.test(msg=f"User {i+1}", user_name=user.get("name", "unknown")):
304
            assert user["name"], "Name cannot be empty"
305
            assert user["age"] > 0, "Age must be positive"
306
            assert "@" in user["email"], "Email must contain @"
307
```
308

309
### Integration with unittest.TestCase
310

311
```python
312
import unittest
313

314
class DataProcessingTests(unittest.TestCase):
315
    def test_process_multiple_files(self):
316
        files = ["data1.txt", "data2.txt", "corrupted.txt", "data4.txt"]
317
        
318
        for filename in files:
319
            with self.subTest(filename=filename):
320
                result = process_file(filename)
321
                self.assertIsNotNone(result)
322
                self.assertGreater(len(result), 0)
323
```
324

325
## Error Handling
326

327
The plugin handles exceptions within subtests gracefully:
328

329
- **Assertion failures**: Each subtest failure is reported individually
330
- **Unexpected exceptions**: Captured and reported with full context
331
- **Test continuation**: Remaining subtests continue to run after failures
332
- **Final test status**: Overall test fails if any subtest fails, but all subtests are executed
333

334
## Output Format
335

336
Subtest failures are reported with clear identification:
337

338
```
339
____________________ test_example [custom message] (i=1) ____________________
340
____________________ test_example [custom message] (i=3) ____________________
341
```
342

343
The format includes:
344
- Test function name
345
- Custom message (if provided)  
346
- Parameter values for identification
347
- Standard pytest failure details
348

349
## Type Support
350

351
The package includes complete type annotations (`py.typed` marker file) for full type checking support with mypy and other type checkers.
352

353
## Type Definitions
354

355
```python { .api }
356
from typing import Any, Generator, Mapping
357
from types import TracebackType
358
from _pytest.fixtures import SubRequest
359
from _pytest.reports import TestReport
360
import pytest
361

362
# Core types used throughout the API
363
Generator[SubTests, None, None]
364
# Generator that yields SubTests instances
365

366
tuple[str, str, str | Mapping[str, bool]] | None  
367
# Return type for pytest_report_teststatus hook
368

369
dict[str, Any]
370
# Generic dictionary type used in serialization and kwargs
371

372
type[Exception] | None
373
# Exception type annotation for context manager
374
```