0
# Stub Tools
1

2
Tools for generating and validating Python stub files (.pyi) for type checking without runtime dependencies. These tools enable type checking of untyped libraries and creating type-only interfaces.
3

4
## Capabilities
5

6
### Stub Generation (stubgen)
7

8
Automatically generate Python stub files from source code or runtime inspection.
9

10
```python { .api }
11
def main(args: list[str] | None = None) -> None:
12
    """
13
    Main entry point for stub generation.
14
    
15
    Parameters:
16
    - args: Command line arguments (uses sys.argv if None)
17
    
18
    Usage:
19
    stubgen.main(['mymodule'])  # Generate stubs for mymodule
20
    """
21
```
22

23
#### Generation Modes
24

25
**Source-based Generation**: Analyze Python source code to create stubs
26
```bash
27
stubgen mymodule.py           # Generate from source file
28
stubgen -p mypackage          # Generate from package source
29
```
30

31
**Runtime-based Generation**: Inspect imported modules at runtime
32
```bash
33
stubgen -m requests           # Generate from installed module
34
stubgen -m numpy pandas       # Generate multiple modules
35
```
36

37
#### Key Features
38

39
- **Preserve public API**: Only exports public names and interfaces
40
- **Type annotation extraction**: Extracts existing type hints from source
41
- **Docstring handling**: Can include or exclude docstrings
42
- **Private member control**: Options for including private definitions
43
- **Cross-references**: Maintains import relationships between modules
44

45
### Stub Testing (stubtest)
46

47
Validate stub files against runtime modules to ensure accuracy and completeness.
48

49
```python { .api }
50
def main(args: list[str] | None = None) -> None:
51
    """
52
    Main entry point for stub validation.
53
    
54
    Parameters:
55
    - args: Command line arguments (uses sys.argv if None)
56
    
57
    Usage:
58
    stubtest.main(['mymodule'])  # Test stubs for mymodule
59
    """
60
```
61

62
#### Validation Types
63

64
**Signature Validation**: Check function and method signatures match
65
```bash
66
stubtest mymodule             # Basic signature checking
67
stubtest --check-typeddict mymodule  # Include TypedDict validation
68
```
69

70
**Completeness Checking**: Ensure all public APIs are stubbed
71
```bash
72
stubtest --ignore-missing-stub mymodule  # Allow missing stubs
73
```
74

75
**Runtime Compatibility**: Verify stubs work with actual runtime behavior
76
```bash
77
stubtest --allowlist allowlist.txt mymodule  # Use allowlist for known issues
78
```
79

80
## Advanced Usage
81

82
### Programmatic Stub Generation
83

84
```python
85
import subprocess
86
import tempfile
87
import os
88
from pathlib import Path
89

90
class StubGenerator:
91
    """Programmatic interface for stub generation."""
92
    
93
    def __init__(self, output_dir: str = "stubs"):
94
        self.output_dir = Path(output_dir)
95
        self.output_dir.mkdir(exist_ok=True)
96
    
97
    def generate_from_source(self, source_paths: list[str]) -> bool:
98
        """Generate stubs from source files."""
99
        cmd = [
100
            'stubgen',
101
            '-o', str(self.output_dir),
102
            '--include-private',
103
            '--no-import'
104
        ] + source_paths
105
        
106
        result = subprocess.run(cmd, capture_output=True, text=True)
107
        return result.returncode == 0
108
    
109
    def generate_from_modules(self, module_names: list[str]) -> bool:
110
        """Generate stubs from installed modules."""
111
        cmd = [
112
            'stubgen',
113
            '-o', str(self.output_dir),
114
            '-m'
115
        ] + module_names
116
        
117
        result = subprocess.run(cmd, capture_output=True, text=True)
118
        return result.returncode == 0
119
    
120
    def generate_package_stubs(self, package_name: str) -> bool:
121
        """Generate stubs for entire package."""
122
        cmd = [
123
            'stubgen',
124
            '-o', str(self.output_dir),
125
            '-p', package_name
126
        ]
127
        
128
        result = subprocess.run(cmd, capture_output=True, text=True)
129
        return result.returncode == 0
130

131
# Usage
132
generator = StubGenerator("my_stubs")
133
generator.generate_from_modules(['requests', 'numpy'])
134
generator.generate_package_stubs('mypackage')
135
```
136

137
### Programmatic Stub Testing
138

139
```python
140
import subprocess
141
import json
142
from pathlib import Path
143

144
class StubTester:
145
    """Programmatic interface for stub testing."""
146
    
147
    def __init__(self, allowlist_file: str | None = None):
148
        self.allowlist_file = allowlist_file
149
    
150
    def test_stubs(self, module_names: list[str]) -> dict:
151
        """Test stubs and return results."""
152
        cmd = ['stubtest'] + module_names
153
        
154
        if self.allowlist_file:
155
            cmd.extend(['--allowlist', self.allowlist_file])
156
        
157
        result = subprocess.run(cmd, capture_output=True, text=True)
158
        
159
        return {
160
            'success': result.returncode == 0,
161
            'stdout': result.stdout,
162
            'stderr': result.stderr,
163
            'errors': self.parse_errors(result.stderr)
164
        }
165
    
166
    def parse_errors(self, stderr: str) -> list[dict]:
167
        """Parse stubtest error output."""
168
        errors = []
169
        for line in stderr.strip().split('\n'):
170
            if 'error:' in line:
171
                # Parse error format: module.function: error: message
172
                parts = line.split(':', 2)  
173
                if len(parts) >= 3:
174
                    errors.append({
175
                        'location': parts[0].strip(),
176
                        'message': parts[2].strip()
177
                    })
178
        return errors
179
    
180
    def generate_allowlist(self, module_names: list[str]) -> str:
181
        """Generate allowlist for current differences."""
182
        cmd = ['stubtest', '--generate-allowlist'] + module_names
183
        result = subprocess.run(cmd, capture_output=True, text=True)
184
        return result.stdout
185

186
# Usage  
187
tester = StubTester('allowlist.txt')
188
results = tester.test_stubs(['mymodule'])
189

190
if not results['success']:
191
    print(f"Found {len(results['errors'])} stub errors")
192
    for error in results['errors']:
193
        print(f"  {error['location']}: {error['message']}")
194
```
195

196
### Integration with Build Systems
197

198
```python
199
# setup.py integration
200
from setuptools import setup
201
from setuptools.command.build import build
202
import subprocess
203
import os
204

205
class BuildWithStubs(build):
206
    """Custom build command that generates stubs."""
207
    
208
    def run(self):
209
        # Run normal build
210
        super().run()
211
        
212
        # Generate stubs for the package
213
        if self.should_generate_stubs():
214
            self.generate_stubs()
215
    
216
    def should_generate_stubs(self) -> bool:
217
        """Check if stubs should be generated."""
218
        return os.environ.get('GENERATE_STUBS', '').lower() == 'true'
219
    
220
    def generate_stubs(self):
221
        """Generate stubs for the package."""
222
        package_name = 'mypackage'  # Replace with actual package name
223
        
224
        print("Generating stub files...")
225
        result = subprocess.run([
226
            'stubgen',
227
            '-o', 'stubs',
228
            '-p', package_name
229
        ], capture_output=True, text=True)
230
        
231
        if result.returncode != 0:
232
            print(f"Stub generation failed: {result.stderr}")
233
        else:
234
            print("Stub files generated successfully")
235

236
setup(
237
    name="mypackage",
238
    cmdclass={'build': BuildWithStubs},
239
    # ... other setup parameters
240
)
241
```
242

243
### CI/CD Integration
244

245
```yaml
246
# GitHub Actions workflow for stub validation
247
name: Validate Stubs
248

249
on: [push, pull_request]
250

251
jobs:
252
  test-stubs:
253
    runs-on: ubuntu-latest
254
    
255
    steps:
256
    - uses: actions/checkout@v2
257
    
258
    - name: Set up Python
259
      uses: actions/setup-python@v2
260
      with:
261
        python-version: 3.11
262
    
263
    - name: Install dependencies
264
      run: |
265
        pip install mypy
266
        pip install -e .  # Install the package
267
    
268
    - name: Generate stubs
269
      run: |
270
        stubgen -o stubs -p mypackage
271
    
272
    - name: Test stubs
273
      run: |
274
        stubtest --allowlist stubtest.allowlist mypackage
275
    
276
    - name: Upload stub artifacts
277
      uses: actions/upload-artifact@v2
278
      with:
279
        name: stubs
280
        path: stubs/
281
```
282

283
## Stub File Examples
284

285
### Basic Stub Structure
286

287
```python
288
# mymodule.pyi - Basic stub file
289
from typing import Any, Optional
290

291
def process_data(data: list[Any], options: Optional[dict[str, Any]] = ...) -> dict[str, Any]: ...
292

293
class DataProcessor:
294
    def __init__(self, config: dict[str, Any]) -> None: ...
295
    def process(self, data: Any) -> Any: ...
296
    @property
297
    def status(self) -> str: ...
298

299
ERROR_CODES: dict[str, int]
300
DEFAULT_CONFIG: dict[str, Any]
301
```
302

303
### Advanced Stub Features
304

305
```python
306
# advanced.pyi - Advanced stub features
307
from typing import Generic, TypeVar, Protocol, overload
308
from typing_extensions import ParamSpec, TypeVarTuple
309

310
T = TypeVar('T')
311
P = ParamSpec('P')
312
Ts = TypeVarTuple('Ts')
313

314
class Container(Generic[T]):
315
    def __init__(self, item: T) -> None: ...
316
    def get(self) -> T: ...
317
    def set(self, item: T) -> None: ...
318

319
class Callable(Protocol[P, T]):
320
    def __call__(self, *args: P.args, **kwargs: P.kwargs) -> T: ...
321

322
@overload
323
def convert(value: int) -> str: ...
324
@overload  
325
def convert(value: str) -> int: ...
326
def convert(value: int | str) -> str | int: ...
327

328
# Variadic generic function
329
def process_multiple(*args: *Ts) -> tuple[*Ts]: ...
330
```
331

332
## Best Practices
333

334
### Stub Maintenance Workflow
335

336
```python
337
class StubMaintenanceWorkflow:
338
    """Automated workflow for stub maintenance."""
339
    
340
    def __init__(self, package_name: str):
341
        self.package_name = package_name
342
        self.stubs_dir = Path("stubs")
343
    
344
    def update_stubs(self) -> bool:
345
        """Update stubs and validate them."""
346
        # 1. Regenerate stubs
347
        if not self.regenerate_stubs():
348
            return False
349
        
350
        # 2. Test against runtime
351
        test_results = self.test_stubs()
352
        if not test_results['success']:
353
            # 3. Update allowlist if needed
354
            self.update_allowlist(test_results['errors'])
355
        
356
        # 4. Final validation
357
        return self.validate_stubs()
358
    
359
    def regenerate_stubs(self) -> bool:
360
        """Regenerate stub files."""
361
        cmd = [
362
            'stubgen',
363
            '-o', str(self.stubs_dir),
364
            '-p', self.package_name,
365
            '--include-private'
366
        ]
367
        
368
        result = subprocess.run(cmd, capture_output=True)
369
        return result.returncode == 0
370
    
371
    def test_stubs(self) -> dict:
372
        """Test stub accuracy."""
373
        tester = StubTester()
374
        return tester.test_stubs([self.package_name])
375
    
376
    def update_allowlist(self, errors: list[dict]):
377
        """Update allowlist based on test errors."""
378
        allowlist_entries = []
379
        
380
        for error in errors:
381
            if self.should_allow_error(error):
382
                allowlist_entries.append(error['location'])
383
        
384
        # Write updated allowlist
385
        with open('stubtest.allowlist', 'w') as f:
386
            f.write('\n'.join(allowlist_entries))
387
    
388
    def should_allow_error(self, error: dict) -> bool:
389
        """Determine if error should be allowlisted."""
390
        message = error['message'].lower()
391
        
392
        # Common patterns to allowlist
393
        allowlist_patterns = [
394
            'runtime argument',  # Runtime-only arguments
395
            'is not present at runtime',  # Stub-only definitions
396
            'incompatible default',  # Different default values
397
        ]
398
        
399
        return any(pattern in message for pattern in allowlist_patterns)
400
    
401
    def validate_stubs(self) -> bool:
402
        """Final validation of stub files."""
403
        # Run mypy on stub files themselves
404
        cmd = ['mypy', str(self.stubs_dir)]
405
        result = subprocess.run(cmd, capture_output=True)
406
        return result.returncode == 0
407

408
# Usage
409
workflow = StubMaintenanceWorkflow('mypackage')
410
success = workflow.update_stubs()
411
```
412

413
### Quality Assurance
414

415
```python
416
def validate_stub_quality(stub_file: Path) -> dict:
417
    """Validate stub file quality and completeness."""
418
    with open(stub_file) as f:
419
        content = f.read()
420
    
421
    issues = []
422
    
423
    # Check for common issues
424
    if 'Any' in content:
425
        any_count = content.count('Any')
426
        if any_count > 10:  # Threshold for too many Any types
427
            issues.append(f"Excessive use of Any ({any_count} occurrences)")
428
    
429
    if '...' not in content:
430
        issues.append("Missing ellipsis in function bodies")
431
    
432
    # Check for proper imports
433
    required_imports = []
434
    if 'Optional[' in content and 'from typing import' not in content:
435
        required_imports.append('Optional')
436
    
437
    if required_imports:
438
        issues.append(f"Missing imports: {', '.join(required_imports)}")
439
    
440
    return {
441
        'file': str(stub_file),
442
        'issues': issues,
443
        'quality_score': max(0, 100 - len(issues) * 10)
444
    }
445

446
# Usage
447
quality_report = validate_stub_quality(Path('stubs/mymodule.pyi'))
448
print(f"Quality score: {quality_report['quality_score']}/100")
449
```