0
# Testing Utilities
1

2
Comprehensive testing framework for Sphinx extensions providing HTML/LaTeX regression testing, fixtures, development tools, and testing infrastructure. These utilities enable developers to create robust tests for Sphinx extensions and ensure consistent documentation output.
3

4
## Capabilities
5

6
### Test Application Framework
7

8
Core testing infrastructure for creating and managing Sphinx application instances for testing.
9

10
```python { .api }
11
class SphinxBuilder:
12
    """Builder for testing Sphinx extensions."""
13
    
14
    def __init__(self, **kwargs) -> None: ...
15
    
16
    def build(self, filenames: List[str]) -> None:
17
        """Build specified files."""
18
        
19
    def get_output(self, filename: str) -> str:
20
        """Get built output content for a file."""
21
        
22
    def get_warnings(self) -> List[str]:
23
        """Get build warnings."""
24

25
def make_test_app(**kwargs) -> Sphinx:
26
    """
27
    Create Sphinx application for testing.
28
    
29
    Args:
30
        **kwargs: Configuration options for test app
31
        
32
    Returns:
33
        Configured Sphinx application instance
34
    """
35

36
def run_setup(app: Sphinx) -> None:
37
    """
38
    Run extension setup for testing.
39
    
40
    Args:
41
        app: Sphinx application instance
42
    """
43
```
44

45
### HTML Regression Testing
46

47
Utilities for testing HTML output with regression detection and comparison.
48

49
```python { .api }
50
class HTMLRegressionFixture:
51
    """HTML regression testing utilities."""
52
    
53
    def __init__(self, test_dir: str) -> None:
54
        """
55
        Initialize HTML regression fixture.
56
        
57
        Args:
58
            test_dir: Directory containing test files
59
        """
60
        
61
    def check_output(self, filename: str, expected_file: str) -> None:
62
        """Check HTML output against expected results."""
63
        
64
    def update_expected(self, filename: str) -> None:
65
        """Update expected output file with current results."""
66

67
def check_html_regression(test_output: str, expected_output: str) -> None:
68
    """
69
    Check HTML output against expected results.
70
    
71
    Args:
72
        test_output: Generated HTML output
73
        expected_output: Expected HTML output
74
        
75
    Raises:
76
        AssertionError: If outputs don't match
77
    """
78

79
def normalize_html(html_content: str) -> str:
80
    """
81
    Normalize HTML content for comparison.
82
    
83
    Args:
84
        html_content: Raw HTML content
85
        
86
    Returns:
87
        Normalized HTML content
88
    """
89

90
def remove_html_footer(html_content: str) -> str:
91
    """
92
    Remove Sphinx-generated footer from HTML content.
93
    
94
    Args:
95
        html_content: HTML content with footer
96
        
97
    Returns:
98
        HTML content without footer
99
    """
100
```
101

102
### LaTeX Regression Testing
103

104
Specialized testing utilities for LaTeX output validation and regression detection.
105

106
```python { .api }
107
class LaTeXRegressionFixture:
108
    """LaTeX regression testing utilities."""
109
    
110
    def __init__(self, test_dir: str) -> None:
111
        """
112
        Initialize LaTeX regression fixture.
113
        
114
        Args:
115
            test_dir: Directory containing test files
116
        """
117
        
118
    def check_output(self, filename: str, expected_file: str) -> None:
119
        """Check LaTeX output against expected results."""
120
        
121
    def normalize_latex(self, latex_content: str) -> str:
122
        """Normalize LaTeX content for comparison."""
123

124
def check_latex_regression(test_output: str, expected_output: str) -> None:
125
    """
126
    Check LaTeX output against expected results.
127
    
128
    Args:
129
        test_output: Generated LaTeX output  
130
        expected_output: Expected LaTeX output
131
        
132
    Raises:
133
        AssertionError: If outputs don't match
134
    """
135

136
def normalize_latex_whitespace(latex_content: str) -> str:
137
    """
138
    Normalize whitespace in LaTeX content.
139
    
140
    Args:
141
        latex_content: Raw LaTeX content
142
        
143
    Returns:
144
        LaTeX content with normalized whitespace
145
    """
146
```
147

148
### Asset Testing
149

150
Utilities for testing asset copying and management in Sphinx extensions.
151

152
```python { .api }
153
def check_asset_copy(app: Sphinx, asset_name: str, expected_location: str) -> None:
154
    """
155
    Verify that assets are copied to the correct location.
156
    
157
    Args:
158
        app: Sphinx application instance
159
        asset_name: Name of asset file
160
        expected_location: Expected location in build output
161
        
162
    Raises:
163
        AssertionError: If asset is not found in expected location
164
    """
165

166
def verify_css_inclusion(html_output: str, css_file: str) -> None:
167
    """
168
    Verify that CSS file is included in HTML output.
169
    
170
    Args:
171
        html_output: Generated HTML content
172
        css_file: CSS filename to check for
173
        
174
    Raises:
175
        AssertionError: If CSS file is not included
176
    """
177

178
def check_js_inclusion(html_output: str, js_file: str) -> None:
179
    """
180
    Verify that JavaScript file is included in HTML output.
181
    
182
    Args:
183
        html_output: Generated HTML content
184
        js_file: JavaScript filename to check for
185
        
186
    Raises:
187
        AssertionError: If JavaScript file is not included
188
    """
189
```
190

191
### Configuration Testing
192

193
Testing utilities for Sphinx configuration validation and processing.
194

195
```python { .api }
196
class ConfigTester:
197
    """Utilities for testing Sphinx configuration."""
198
    
199
    def __init__(self, base_config: Dict[str, Any]) -> None:
200
        """
201
        Initialize configuration tester.
202
        
203
        Args:
204
            base_config: Base configuration dictionary
205
        """
206
        
207
    def test_config_value(self, key: str, value: Any, should_pass: bool = True) -> None:
208
        """Test configuration value validation."""
209
        
210
    def test_missing_required(self, required_keys: List[str]) -> None:
211
        """Test behavior with missing required configuration."""
212

213
def test_config_validation(config: Dict[str, Any], validator_func: Callable) -> None:
214
    """
215
    Test configuration validation function.
216
    
217
    Args:
218
        config: Configuration dictionary to test
219
        validator_func: Validation function to test
220
        
221
    Raises:
222
        AssertionError: If validation behaves unexpectedly
223
    """
224
```
225

226
### Extension Testing
227

228
Base classes and utilities for testing Sphinx extensions.
229

230
```python { .api }
231
class SphinxTestCase:
232
    """Base class for Sphinx extension testing."""
233
    
234
    def setup_app(self, **kwargs) -> Sphinx:
235
        """
236
        Set up test Sphinx application.
237
        
238
        Args:
239
            **kwargs: Configuration options
240
            
241
        Returns:
242
            Configured Sphinx application
243
        """
244
        
245
    def build_docs(self, builder: str = 'html') -> None:
246
        """
247
        Build documentation for testing.
248
        
249
        Args:
250
            builder: Builder name to use
251
        """
252
        
253
    def get_output(self, filename: str) -> str:
254
        """
255
        Get built output content.
256
        
257
        Args:
258
            filename: Output filename to retrieve
259
            
260
        Returns:
261
            File content as string
262
        """
263
        
264
    def assert_build_succeeds(self) -> None:
265
        """Assert that documentation build succeeds."""
266
        
267
    def assert_no_warnings(self) -> None:
268
        """Assert that build produces no warnings."""
269

270
class ExtensionTestCase(SphinxTestCase):
271
    """Base class for testing individual extensions."""
272
    
273
    extension_name: str = None
274
    
275
    def test_setup(self) -> None:
276
        """Test extension setup process."""
277
        
278
    def test_configuration(self) -> None:
279
        """Test extension configuration handling."""
280
```
281

282
### Directive and Role Testing
283

284
Specialized testing utilities for Sphinx directives and roles.
285

286
```python { .api }
287
def test_directive(directive_class: Type[SphinxDirective], content: List[str], 
288
                  options: Dict[str, Any] = {}) -> List[Node]:
289
    """
290
    Test directive processing.
291
    
292
    Args:
293
        directive_class: Directive class to test
294
        content: Directive content lines
295
        options: Directive options
296
        
297
    Returns:
298
        Generated nodes from directive
299
    """
300

301
def test_role(role_func: Callable, text: str, options: Dict[str, Any] = {}) -> Tuple[List[Node], List[system_message]]:
302
    """
303
    Test role processing.
304
    
305
    Args:
306
        role_func: Role function to test
307
        text: Role text content
308
        options: Role options
309
        
310
    Returns:
311
        Tuple of generated nodes and messages
312
    """
313

314
class DirectiveTestCase:
315
    """Test case for directive testing."""
316
    
317
    def test_directive_parsing(self, directive_class: Type[SphinxDirective]) -> None:
318
        """Test directive argument and option parsing."""
319
        
320
    def test_directive_output(self, directive_class: Type[SphinxDirective]) -> None:
321
        """Test directive node generation."""
322
```
323

324
### Mock and Fixture Utilities
325

326
Utilities for creating mocks and fixtures for testing.
327

328
```python { .api }
329
class MockApp:
330
    """Mock Sphinx application for lightweight testing."""
331
    
332
    def __init__(self, config: Dict[str, Any] = {}) -> None: ...
333
    def setup_extension(self, extension: str) -> None: ...
334
    def add_config_value(self, name: str, default: Any, rebuild: str) -> None: ...
335

336
class MockBuilder:
337
    """Mock builder for testing build processes."""
338
    
339
    def __init__(self, name: str = 'html') -> None: ...
340
    def read_doc(self, docname: str) -> None: ...
341
    def write_doc(self, docname: str, doctree: Element) -> None: ...
342

343
def create_test_source_dir(content_map: Dict[str, str]) -> str:
344
    """
345
    Create temporary source directory with test content.
346
    
347
    Args:
348
        content_map: Mapping of filenames to content
349
        
350
    Returns:
351
        Path to created temporary directory
352
    """
353

354
def cleanup_test_dir(test_dir: str) -> None:
355
    """
356
    Clean up temporary test directory.
357
    
358
    Args:
359
        test_dir: Directory path to clean up
360
    """
361
```
362

363
### Integration Testing
364

365
Utilities for end-to-end integration testing of Sphinx extensions.
366

367
```python { .api }
368
def run_integration_test(extension_name: str, test_docs_dir: str, 
369
                        expected_outputs: Dict[str, str]) -> None:
370
    """
371
    Run full integration test for extension.
372
    
373
    Args:
374
        extension_name: Name of extension to test
375
        test_docs_dir: Directory containing test documentation
376
        expected_outputs: Expected output files and their content
377
        
378
    Raises:
379
        AssertionError: If integration test fails
380
    """
381

382
class IntegrationTestSuite:
383
    """Test suite for integration testing."""
384
    
385
    def __init__(self, extensions: List[str]) -> None: ...
386
    def add_test_case(self, name: str, source_files: Dict[str, str]) -> None: ...
387
    def run_all_tests(self) -> None: ...
388

389
def compare_build_outputs(output_dir1: str, output_dir2: str, 
390
                         ignore_patterns: List[str] = []) -> bool:
391
    """
392
    Compare two build output directories.
393
    
394
    Args:
395
        output_dir1: First output directory
396
        output_dir2: Second output directory  
397
        ignore_patterns: File patterns to ignore in comparison
398
        
399
    Returns:
400
        True if directories match, False otherwise
401
    """
402
```
403

404
### Performance Testing
405

406
Utilities for performance testing and benchmarking Sphinx extensions.
407

408
```python { .api }
409
def benchmark_build_time(app: Sphinx, iterations: int = 5) -> float:
410
    """
411
    Benchmark documentation build time.
412
    
413
    Args:
414
        app: Sphinx application to benchmark
415
        iterations: Number of iterations to run
416
        
417
    Returns:
418
        Average build time in seconds
419
    """
420

421
def profile_extension_setup(extension_name: str) -> Dict[str, float]:
422
    """
423
    Profile extension setup performance.
424
    
425
    Args:
426
        extension_name: Extension to profile
427
        
428
    Returns:
429
        Dictionary of timing information
430
    """
431

432
class PerformanceTestCase:
433
    """Test case for performance testing."""
434
    
435
    def test_build_performance(self, max_time: float) -> None:
436
        """Test that build completes within specified time."""
437
        
438
    def test_memory_usage(self, max_memory: int) -> None:
439
        """Test maximum memory usage during build."""
440
```
441

442
### Error Testing
443

444
Utilities for testing error handling and edge cases.
445

446
```python { .api }
447
def test_error_handling(func: Callable, error_cases: List[Tuple[Any, Type[Exception]]]) -> None:
448
    """
449
    Test function error handling.
450
    
451
    Args:
452
        func: Function to test
453
        error_cases: List of (input, expected_exception) tuples
454
    """
455

456
def assert_warning_raised(warning_class: Type[Warning], func: Callable, *args, **kwargs) -> None:
457
    """
458
    Assert that specific warning is raised.
459
    
460
    Args:
461
        warning_class: Warning class to expect
462
        func: Function that should raise warning
463
        *args: Function arguments
464
        **kwargs: Function keyword arguments
465
    """
466

467
class ErrorTestCase:
468
    """Test case for error handling testing."""
469
    
470
    def test_missing_dependencies(self) -> None:
471
        """Test behavior with missing dependencies."""
472
        
473
    def test_invalid_configuration(self) -> None:
474
        """Test handling of invalid configuration."""
475
        
476
    def test_malformed_input(self) -> None:
477
        """Test handling of malformed input."""
478
```