0
# Source Code Parsing and Analysis
1
2
Static analysis of Python source code using astroid to extract API information without importing modules. AutoAPI's parser handles complex parsing scenarios including type annotations, decorators, and inheritance hierarchies.
3
4
## Core Parser
5
6
### Parser Class
7
8
```python { .api }
9
class Parser:
10
"""
11
Main parser for Python source code using astroid static analysis.
12
13
The Parser converts Python source files into structured data without
14
requiring code execution or imports, making it safe for documenting
15
packages with complex dependencies.
16
"""
17
18
def parse_file(self, file_path: str):
19
"""
20
Parse a single Python source file.
21
22
Args:
23
file_path (str): Absolute path to Python file to parse
24
25
Returns:
26
list: List of parsed Python objects from the file
27
28
Raises:
29
astroid.AstroidError: If file cannot be parsed
30
"""
31
32
def parse_file_in_namespace(self, file_path: str, dir_root: str):
33
"""
34
Parse a file within a namespace context for proper import resolution.
35
36
Args:
37
file_path (str): Path to file being parsed
38
dir_root (str): Root directory for namespace resolution
39
40
Returns:
41
list: Parsed objects with proper namespace context
42
43
This method ensures proper module path resolution for packages
44
and nested modules.
45
"""
46
47
def parse(self, node):
48
"""
49
Parse an astroid AST node into AutoAPI objects.
50
51
Args:
52
node: Astroid AST node to parse
53
54
Returns:
55
list: AutoAPI objects representing the parsed node
56
57
Handles all Python constructs: modules, classes, functions,
58
methods, properties, attributes, and data.
59
"""
60
```
61
62
From `autoapi._parser`:
63
```python
64
from autoapi._parser import Parser
65
```
66
67
## Parsing Capabilities
68
69
### Supported Python Constructs
70
71
```python { .api }
72
# Module-level parsing
73
"""Extracts module docstrings, imports, and all module-level definitions."""
74
75
# Class parsing with inheritance
76
"""Parses class definitions including:
77
- Base classes and inheritance hierarchy
78
- Class docstrings and decorators
79
- All class methods and properties
80
- Class-level attributes and data
81
- Nested classes
82
"""
83
84
# Function and method parsing
85
"""Extracts function information including:
86
- Function signatures with type annotations
87
- Parameter names, types, and default values
88
- Return type annotations
89
- Decorators applied to functions
90
- Docstrings with parameter documentation
91
"""
92
93
# Property parsing
94
"""Handles property decorators and descriptors:
95
- @property decorated methods
96
- Getter, setter, deleter methods
97
- Property docstrings
98
"""
99
100
# Data and attribute parsing
101
"""Extracts module and class-level data:
102
- Variable assignments with type annotations
103
- Constants and configuration values
104
- Import statements and aliases
105
"""
106
```
107
108
### Advanced Parsing Features
109
110
```python { .api }
111
def parse_assign(self, node):
112
"""
113
Parse assignment statements including type annotations.
114
115
Args:
116
node: astroid.Assign or astroid.AnnAssign node
117
118
Returns:
119
list: PythonData or PythonAttribute objects
120
121
Handles:
122
- Simple assignments: x = value
123
- Annotated assignments: x: int = value
124
- Multiple assignments: a, b = values
125
- Class and instance attributes
126
"""
127
128
def parse_annassign(self, node):
129
"""
130
Parse annotated assignment statements (PEP 526).
131
132
Args:
133
node: astroid.AnnAssign node
134
135
Returns:
136
list: Parsed objects with type information
137
138
Extracts type annotations for variables without requiring imports.
139
"""
140
```
141
142
## Docstring Processing
143
144
### Docstring Utilities
145
146
```python { .api }
147
def _prepare_docstring(doc):
148
"""
149
Prepare a docstring for documentation rendering.
150
151
Args:
152
doc (str): Raw docstring from source code
153
154
Returns:
155
str: Formatted docstring ready for Sphinx processing
156
157
Applies Sphinx docstring utilities:
158
- Normalizes indentation
159
- Handles reStructuredText formatting
160
- Preserves code blocks and examples
161
"""
162
```
163
164
## Astroid Utilities
165
166
AutoAPI includes comprehensive utilities for working with astroid AST nodes:
167
168
### Argument Processing
169
170
```python { .api }
171
class ArgInfo:
172
"""
173
Named tuple for structured function argument information.
174
175
Fields:
176
prefix (str): Argument prefix ('', '*', '**')
177
name (str): Parameter name
178
annotation (str): Type annotation if present
179
default_value (str): Default value if present
180
"""
181
182
def _format_args(args_info, include_annotations=True, ignore_self=None):
183
"""
184
Format function arguments for display in documentation.
185
186
Args:
187
args_info: List of ArgInfo objects
188
include_annotations (bool): Whether to include type annotations
189
ignore_self (str): Parameter name to ignore (typically 'self')
190
191
Returns:
192
str: Formatted argument string for function signatures
193
"""
194
```
195
196
### Import Resolution
197
198
```python { .api }
199
def resolve_import_alias(name, import_names):
200
"""
201
Resolve aliased import names to their original names.
202
203
Args:
204
name (str): Name to resolve
205
import_names (dict): Mapping of aliases to original names
206
207
Returns:
208
str: Resolved name or original if no alias found
209
"""
210
211
def get_full_import_name(import_from, name):
212
"""
213
Get full import path from ImportFrom node.
214
215
Args:
216
import_from: astroid.ImportFrom node
217
name (str): Imported name
218
219
Returns:
220
str: Full dotted import path
221
"""
222
223
def resolve_qualname(node, basename):
224
"""
225
Resolve fully qualified names for nested objects.
226
227
Args:
228
node: astroid node
229
basename (str): Base name for qualification
230
231
Returns:
232
str: Fully qualified name including module path
233
"""
234
```
235
236
### Value Extraction
237
238
```python { .api }
239
def get_assign_value(node):
240
"""
241
Extract assignment values from AST nodes.
242
243
Args:
244
node: astroid assignment node
245
246
Returns:
247
str: String representation of assigned value
248
249
Handles constants, expressions, and complex values safely.
250
"""
251
252
def get_assign_annotation(node):
253
"""
254
Get type annotations from assignment statements.
255
256
Args:
257
node: astroid assignment node
258
259
Returns:
260
str: Type annotation string or None
261
"""
262
263
def get_const_value(node):
264
"""
265
Extract constant values from AST nodes.
266
267
Args:
268
node: astroid node containing constant
269
270
Returns:
271
str: String representation of constant value
272
273
Safely extracts string, number, and boolean constants.
274
"""
275
```
276
277
### Special Construct Detection
278
279
```python { .api }
280
def is_constructor(node):
281
"""
282
Check if a node represents a constructor method.
283
284
Args:
285
node: astroid method node
286
287
Returns:
288
bool: True if node is __init__ or __new__ method
289
"""
290
291
def is_functional_namedtuple(node):
292
"""
293
Check if a node represents a functional namedtuple definition.
294
295
Args:
296
node: astroid assignment node
297
298
Returns:
299
bool: True if assignment creates namedtuple via function call
300
"""
301
```
302
303
## Error Handling
304
305
The parser includes robust error handling for common parsing issues:
306
307
```python { .api }
308
# Syntax errors in source files
309
"""Parser logs syntax errors and continues processing other files."""
310
311
# Import resolution failures
312
"""Unresolvable imports are noted but don't halt documentation generation."""
313
314
# Complex type annotations
315
"""Advanced typing constructs are captured as strings when full resolution isn't possible."""
316
317
# Encoding issues
318
"""Source files with encoding problems are handled gracefully."""
319
```
320
321
## Usage Example
322
323
```python { .api }
324
from autoapi._parser import Parser
325
326
# Create parser instance
327
parser = Parser()
328
329
# Parse a single file
330
objects = parser.parse_file('/path/to/module.py')
331
332
# Parse file with namespace context
333
objects = parser.parse_file_in_namespace(
334
'/path/to/package/module.py',
335
'/path/to/package'
336
)
337
338
# Objects contain structured data ready for documentation generation
339
for obj in objects:
340
print(f"{obj.name}: {obj.type}")
341
```
342
343
The parser forms the foundation of AutoAPI's static analysis capabilities, enabling reliable documentation generation without code execution risks.