0
# AST and Node System
1

2
Abstract syntax tree representation and node types used internally by mypy for code analysis. This system provides structured access to Python code for type checking and static analysis.
3

4
## Capabilities
5

6
### Base Node Classes
7

8
Foundation classes for the AST hierarchy providing location information and visitor pattern support.
9

10
```python { .api }
11
class Context:
12
    """
13
    Base class providing source location information for error reporting.
14
    
15
    All AST nodes inherit location tracking for precise error messages
16
    and diagnostic information.
17
    
18
    Attributes:
19
    - line: int - Line number in source file (1-based)
20
    - column: int - Column number in source line (0-based)
21
    """
22

23
class Node(Context):
24
    """
25
    Abstract base class for all AST nodes.
26
    
27
    Root of the AST hierarchy implementing the visitor pattern
28
    for traversal and transformation operations.
29
    
30
    Methods:
31
    - accept(visitor: NodeVisitor[T]) -> T
32
    """
33

34
class Statement(Node):
35
    """
36
    Base class for statement AST nodes.
37
    
38
    Represents executable statements like assignments, function calls,
39
    control flow statements, and declarations.
40
    """
41

42
class Expression(Node):  
43
    """
44
    Base class for expression AST nodes.
45
    
46
    Represents expressions that evaluate to values, including
47
    literals, names, operations, and function calls.
48
    """
49
```
50

51
### File and Module Nodes
52

53
Top-level nodes representing entire source files and modules.
54

55
```python { .api }
56
class MypyFile(Node):
57
    """
58
    Represents an entire source file in mypy's AST.
59
    
60
    Top-level container for all definitions, imports, and statements
61
    in a Python source file.
62
    
63
    Attributes:
64
    - defs: list[Statement] - Top-level definitions and statements
65
    - names: SymbolTable - Module-level symbol table
66
    - imports: list[ImportBase] - Import statements
67
    - is_bom: bool - Whether file starts with BOM
68
    - path: str - File path
69
    - module: str - Module name
70
    """
71
```
72

73
### Class and Function Definition Nodes
74

75
Core definition nodes for classes, functions, and methods.
76

77
```python { .api }
78
class ClassDef(Statement):
79
    """
80
    Represents class definitions.
81
    
82
    Contains class body, inheritance information, decorators,
83
    and metadata for type checking.
84
    
85
    Attributes:
86
    - name: str - Class name
87
    - defs: Block - Class body statements
88
    - base_type_exprs: list[Expression] - Base class expressions
89
    - decorators: list[Expression] - Class decorators
90
    - info: TypeInfo - Semantic information about the class
91
    - type_vars: list[TypeVarLikeType] - Generic type parameters
92
    """
93

94
class FuncDef(Statement):
95
    """
96
    Represents function and method definitions.
97
    
98
    Contains function signature, body, decorators, and type information
99
    for both regular functions and methods.
100
    
101
    Attributes:
102
    - name: str - Function name
103
    - arguments: Arguments - Parameter information
104
    - body: Block - Function body statements
105
    - type: Type | None - Function type annotation
106
    - info: TypeInfo | None - Class info (for methods)
107
    - is_property: bool - Whether this is a property
108
    - is_class_method: bool - Whether this is a classmethod
109
    - is_static_method: bool - Whether this is a staticmethod
110
    """
111

112
class Arguments(Node):
113
    """
114
    Represents function parameter list.
115
    
116
    Contains detailed information about function parameters including
117
    types, defaults, and parameter kinds.
118
    
119
    Attributes:
120
    - arguments: list[Argument] - Individual parameter definitions
121
    - arg_names: list[str] - Parameter names
122
    - arg_kinds: list[int] - Parameter kinds (positional, keyword, etc.)
123
    - initializers: list[Expression | None] - Default value expressions
124
    """
125

126
class Argument(Node):
127
    """
128
    Represents a single function parameter.
129
    
130
    Attributes:
131
    - variable: Var - Variable representing the parameter
132
    - type_annotation: Expression | None - Type annotation
133
    - initializer: Expression | None - Default value
134
    - kind: int - Parameter kind (ARG_POS, ARG_OPT, ARG_STAR, ARG_STAR2)
135
    """
136
```
137

138
### Symbol Table and Type Information
139

140
Core semantic analysis nodes for name resolution and type information.
141

142
```python { .api }
143
class TypeInfo(Node):
144
    """
145
    Contains semantic information about classes.
146
    
147
    Central repository for class metadata including inheritance hierarchy,
148
    method resolution order, and member information.
149
    
150
    Attributes:
151
    - names: SymbolTable - Class member symbol table
152
    - mro: list[TypeInfo] - Method resolution order
153
    - bases: list[Instance] - Direct base classes
154
    - abstract_attributes: list[str] - Abstract method/property names
155
    - is_abstract: bool - Whether class is abstract
156
    - is_protocol: bool - Whether class is a Protocol
157
    - fallback_to_any: bool - Whether to fallback to Any
158
    """
159

160
class SymbolTable(dict[str, SymbolTableNode]):
161
    """
162
    Maps names to their definitions within a scope.
163
    
164
    Used for name resolution during semantic analysis and type checking.
165
    Inherits from dict for convenient name lookup operations.
166
    """
167

168
class SymbolTableNode:
169
    """
170
    Individual entries in symbol tables.
171
    
172
    Links names to their AST nodes, types, and import information.
173
    
174
    Attributes:
175
    - kind: int - Symbol kind (LDEF, GDEF, MDEF, etc.)
176
    - node: Node | None - AST node for the symbol
177
    - type_override: Type | None - Explicit type override
178
    - module_public: bool - Whether symbol is publicly exported
179
    - implicit: bool - Whether symbol was implicitly created
180
    """
181

182
class Var(Node):
183
    """
184
    Represents variable declarations and references.
185
    
186
    Used for local variables, instance variables, class variables,
187
    and global variables.
188
    
189
    Attributes:
190
    - name: str - Variable name
191
    - type: Type | None - Variable type
192
    - is_self: bool - Whether this is 'self' parameter
193
    - is_cls: bool - Whether this is 'cls' parameter
194
    - is_ready: bool - Whether variable is fully analyzed
195
    """
196
```
197

198
### Expression Nodes
199

200
Nodes representing different types of expressions and operations.
201

202
```python { .api }
203
class NameExpr(Expression):
204
    """
205
    Represents name references (variable access).
206
    
207
    Used when accessing variables, functions, classes, or other names.
208
    
209
    Attributes:
210
    - name: str - Name being referenced
211
    - kind: int - Name kind (LDEF, GDEF, MDEF, etc.)
212
    - node: Node | None - Definition node for the name
213
    - fullname: str | None - Fully qualified name
214
    """
215

216
class MemberExpr(Expression):
217
    """
218
    Represents attribute access (obj.attr).
219
    
220
    Used for accessing attributes, methods, and properties of objects.
221
    
222
    Attributes:
223
    - expr: Expression - Object being accessed
224
    - name: str - Attribute name
225
    - kind: int | None - Member kind
226
    - fullname: str | None - Fully qualified attribute name
227
    """
228

229
class CallExpr(Expression):
230
    """
231
    Represents function and method calls.
232
    
233
    Contains call target, arguments, and metadata for type checking
234
    function calls and method invocations.
235
    
236
    Attributes:
237
    - callee: Expression - Function/method being called
238
    - args: list[Expression] - Positional and keyword arguments
239
    - arg_names: list[str | None] - Keyword argument names
240
    - arg_kinds: list[int] - Argument kinds
241
    """
242

243
class IndexExpr(Expression):
244
    """
245
    Represents indexing operations (obj[key]).
246
    
247
    Used for list/dict access, generic type instantiation,
248
    and other subscript operations.
249
    
250
    Attributes:
251
    - base: Expression - Object being indexed
252
    - index: Expression - Index/key expression
253
    - method: str | None - Special method name (__getitem__, etc.)
254
    """
255

256
class SliceExpr(Expression):
257
    """
258
    Represents slicing operations (obj[start:end:step]).
259
    
260
    Attributes:
261
    - begin_index: Expression | None - Start index
262
    - end_index: Expression | None - End index  
263
    - stride: Expression | None - Step value
264
    """
265

266
class OpExpr(Expression):
267
    """
268
    Represents binary operations (x + y, x == y, etc.).
269
    
270
    Attributes:
271
    - op: str - Operator string ('+', '==', 'and', etc.)
272
    - left: Expression - Left operand
273
    - right: Expression - Right operand
274
    - method: str | None - Special method name (__add__, __eq__, etc.)
275
    """
276

277
class UnaryExpr(Expression):
278
    """
279
    Represents unary operations (-x, not x, etc.).
280
    
281
    Attributes:
282
    - op: str - Operator string ('-', 'not', '~', etc.)
283
    - expr: Expression - Operand expression
284
    - method: str | None - Special method name (__neg__, etc.)
285
    """
286
```
287

288
### Literal and Collection Expressions
289

290
Nodes for literal values and collection constructors.
291

292
```python { .api }
293
class IntExpr(Expression):
294
    """
295
    Represents integer literals.
296
    
297
    Attributes:
298
    - value: int - Integer value
299
    """
300

301
class StrExpr(Expression):
302
    """
303
    Represents string literals.
304
    
305
    Attributes:
306
    - value: str - String value
307
    """
308

309
class FloatExpr(Expression):
310
    """
311
    Represents floating-point literals.
312
    
313
    Attributes:
314
    - value: float - Float value
315
    """
316

317
class ListExpr(Expression):
318
    """
319
    Represents list literals [1, 2, 3].
320
    
321
    Attributes:
322
    - items: list[Expression] - List elements
323
    """
324

325
class DictExpr(Expression):
326
    """
327
    Represents dictionary literals {'a': 1, 'b': 2}.
328
    
329
    Attributes:
330
    - items: list[tuple[Expression | None, Expression]] - Key-value pairs
331
    """
332

333
class SetExpr(Expression):
334
    """
335
    Represents set literals {1, 2, 3}.
336
    
337
    Attributes:
338
    - items: list[Expression] - Set elements
339
    """
340

341
class TupleExpr(Expression):
342
    """
343
    Represents tuple literals (1, 2, 3).
344
    
345
    Attributes:
346
    - items: list[Expression] - Tuple elements
347
    """
348
```
349

350
### Statement Nodes
351

352
Nodes representing different types of statements and control flow.
353

354
```python { .api }
355
class AssignmentStmt(Statement):
356
    """
357
    Represents assignment statements (x = y).
358
    
359
    Handles both simple assignments and complex assignment patterns
360
    including tuple unpacking and multiple targets.
361
    
362
    Attributes:
363
    - lvalues: list[Expression] - Left-hand side expressions
364
    - rvalue: Expression - Right-hand side value
365
    - type: Type | None - Optional type annotation
366
    """
367

368
class IfStmt(Statement):
369
    """
370
    Represents if statements and elif chains.
371
    
372
    Attributes:
373
    - expr: list[Expression] - Condition expressions for if/elif
374
    - body: list[Block] - Statement blocks for each condition
375
    - else_body: Block | None - Else block if present
376
    """
377

378
class ForStmt(Statement):
379
    """
380
    Represents for loops.
381
    
382
    Attributes:
383
    - index: Expression - Loop variable(s)
384
    - expr: Expression - Iterable expression
385
    - body: Block - Loop body
386
    - else_body: Block | None - Else block if present
387
    """
388

389
class WhileStmt(Statement):
390
    """
391
    Represents while loops.
392
    
393
    Attributes:
394
    - expr: Expression - Loop condition
395
    - body: Block - Loop body
396
    - else_body: Block | None - Else block if present
397
    """
398

399
class ReturnStmt(Statement):
400
    """
401
    Represents return statements.
402
    
403
    Attributes:
404
    - expr: Expression | None - Return value expression
405
    """
406

407
class RaiseStmt(Statement):
408
    """
409
    Represents raise statements.
410
    
411
    Attributes:
412
    - expr: Expression | None - Exception expression
413
    - from_expr: Expression | None - Chained exception
414
    """
415

416
class TryStmt(Statement):
417
    """
418
    Represents try/except/finally statements.
419
    
420
    Attributes:
421
    - body: Block - Try block
422
    - handlers: list[ExceptHandler] - Exception handlers
423
    - orelse: Block | None - Else block
424
    - finally_body: Block | None - Finally block
425
    """
426

427
class ExceptHandler(Node):
428
    """
429
    Represents individual except clauses.
430
    
431
    Attributes:
432
    - type: Expression | None - Exception type
433
    - name: str | None - Exception variable name
434
    - body: Block - Handler body
435
    """
436
```
437

438
### Import Statements
439

440
Nodes for import and from-import statements.
441

442
```python { .api }
443
class ImportStmt(Statement):
444
    """
445
    Represents import statements (import x, import y as z).
446
    
447
    Attributes:
448
    - ids: list[str] - Module names being imported
449
    - names: list[str | None] - Alias names (None if no alias)
450
    """
451

452
class ImportFromStmt(Statement):
453
    """
454
    Represents from-import statements (from x import y).
455
    
456
    Attributes:
457
    - module: str | None - Module name (None for relative imports)
458
    - names: list[tuple[str, str | None]] - (name, alias) pairs
459
    - relative: int - Relative import level (number of dots)
460
    """
461

462
class ImportAllStmt(Statement):
463
    """
464
    Represents star imports (from x import *).
465
    
466
    Attributes:
467
    - module: str - Module name
468
    - relative: int - Relative import level
469
    """
470
```
471

472
## AST Traversal and Manipulation
473

474
### Visitor Pattern
475

476
```python
477
from mypy.visitor import NodeVisitor
478

479
class ASTAnalyzer(NodeVisitor[None]):
480
    """Example AST visitor for analyzing code patterns."""
481
    
482
    def __init__(self):
483
        self.function_count = 0
484
        self.class_count = 0
485
        self.import_count = 0
486
    
487
    def visit_func_def(self, node: FuncDef) -> None:
488
        """Visit function definitions."""
489
        self.function_count += 1
490
        print(f"Found function: {node.name}")
491
        
492
        # Visit function body
493
        super().visit_func_def(node)
494
    
495
    def visit_class_def(self, node: ClassDef) -> None:
496
        """Visit class definitions."""
497
        self.class_count += 1
498
        print(f"Found class: {node.name}")
499
        
500
        # Analyze base classes
501
        for base in node.base_type_exprs:
502
            print(f"  Base class: {base}")
503
        
504
        # Visit class body
505
        super().visit_class_def(node)
506
    
507
    def visit_import_stmt(self, node: ImportStmt) -> None:
508
        """Visit import statements."""
509
        self.import_count += 1
510
        for module, alias in zip(node.ids, node.names):
511
            if alias:
512
                print(f"Import: {module} as {alias}")
513
            else:
514
                print(f"Import: {module}")
515

516
# Usage
517
analyzer = ASTAnalyzer()
518
mypy_file.accept(analyzer)
519

520
print(f"Summary: {analyzer.function_count} functions, "
521
      f"{analyzer.class_count} classes, {analyzer.import_count} imports")
522
```
523

524
### AST Construction
525

526
```python
527
from mypy.nodes import (
528
    FuncDef, Arguments, Argument, Var, Block, ReturnStmt,
529
    NameExpr, StrExpr, CallExpr
530
)
531

532
def create_function_node(name: str, return_type_name: str) -> FuncDef:
533
    """Create a function AST node programmatically."""
534
    
535
    # Create function arguments
536
    args = Arguments(
537
        arguments=[],
538
        arg_names=[],
539
        arg_kinds=[],
540
        initializers=[]
541
    )
542
    
543
    # Create return statement  
544
    return_stmt = ReturnStmt(StrExpr("Hello, World!"))
545
    
546
    # Create function body
547
    body = Block([return_stmt])
548
    
549
    # Create function definition
550
    func_def = FuncDef(
551
        name=name,
552
        arguments=args,
553
        body=body,
554
        type=None,  # Type will be inferred
555
        type_annotation=NameExpr(return_type_name)
556
    )
557
    
558
    return func_def
559

560
# Usage
561
hello_func = create_function_node("hello", "str")
562
```
563

564
### AST Transformation
565

566
```python
567
from mypy.visitor import NodeTransformer
568
from mypy.nodes import Expression, StrExpr, CallExpr
569

570
class StringLiteralTransformer(NodeTransformer):
571
    """Transform string literals to function calls."""
572
    
573
    def visit_str_expr(self, node: StrExpr) -> Expression:
574
        """Transform string literals to function calls."""
575
        if node.value.startswith("LOG:"):
576
            # Transform "LOG: message" to log_function("message")
577
            message = node.value[4:].strip()
578
            return CallExpr(
579
                callee=NameExpr("log_function"),
580
                args=[StrExpr(message)],
581
                arg_names=[None],
582
                arg_kinds=[ARG_POS]
583
            )
584
        
585
        return node
586

587
# Usage
588
transformer = StringLiteralTransformer()
589
transformed_ast = mypy_file.accept(transformer)
590
```
591

592
## Integration with Type Checking
593

594
### Semantic Analysis Integration
595

596
```python
597
from mypy.semanal import SemanticAnalyzer
598
from mypy.nodes import MypyFile
599

600
def analyze_file_semantics(mypy_file: MypyFile) -> None:
601
    """Perform semantic analysis on AST."""
602
    
603
    # Create semantic analyzer
604
    analyzer = SemanticAnalyzer(
605
        modules={},
606
        missing_modules=set(),
607
        incomplete_type_vars=set(),
608
        options=Options()
609
    )
610
    
611
    # Analyze file
612
    analyzer.visit_mypy_file(mypy_file)
613
    
614
    # Access symbol tables
615
    for name, node in mypy_file.names.items():
616
        print(f"Symbol: {name} -> {node}")
617
```
618

619
### Type Checker Integration
620

621
```python
622
from mypy.checker import TypeChecker
623
from mypy.nodes import Expression
624

625
def check_expression_type(expr: Expression, type_checker: TypeChecker) -> Type:
626
    """Get the type of an expression using the type checker."""
627
    
628
    # Type check the expression
629
    expr_type = expr.accept(type_checker)
630
    
631
    return expr_type
632

633
def validate_function_call(call_expr: CallExpr, type_checker: TypeChecker) -> bool:
634
    """Validate a function call using type information."""
635
    
636
    # Check callee type
637
    callee_type = call_expr.callee.accept(type_checker)
638
    
639
    # Check argument types
640
    arg_types = [arg.accept(type_checker) for arg in call_expr.args]
641
    
642
    # Validate call compatibility
643
    if isinstance(callee_type, CallableType):
644
        # Check argument count and types
645
        return len(arg_types) == len(callee_type.arg_types)
646
    
647
    return False
648
```
649

650
## AST Node Constants
651

652
### Node Kinds and Types
653

654
```python
655
# Symbol table node kinds
656
LDEF = 0  # Local definition
657
GDEF = 1  # Global definition  
658
MDEF = 2  # Class member definition
659
UNBOUND_IMPORTED = 3  # Unbound imported name
660

661
# Argument kinds
662
ARG_POS = 0       # Positional argument
663
ARG_OPT = 1       # Optional argument (with default)
664
ARG_STAR = 2      # *args
665
ARG_STAR2 = 3     # **kwargs
666

667
# Operator mappings
668
op_methods = {
669
    '+': '__add__',
670
    '-': '__sub__',  
671
    '*': '__mul__',
672
    '/': '__truediv__',
673
    '==': '__eq__',
674
    '!=': '__ne__',
675
    '<': '__lt__',
676
    '>': '__gt__',
677
    # ... additional operators
678
}
679
```
680

681
### AST Utilities
682

683
```python
684
def get_qualified_name(node: Node) -> str | None:
685
    """Get fully qualified name for a node."""
686
    if isinstance(node, (FuncDef, ClassDef)):
687
        return node.fullname
688
    elif isinstance(node, NameExpr):
689
        return node.fullname
690
    return None
691

692
def is_method(func_def: FuncDef) -> bool:
693
    """Check if function definition is a method."""
694
    return func_def.info is not None
695

696
def get_method_type(func_def: FuncDef) -> str:
697
    """Get method type (instance, class, static)."""
698
    if func_def.is_class_method:
699
        return "classmethod"
700
    elif func_def.is_static_method:
701
        return "staticmethod"
702
    elif func_def.is_property:
703
        return "property"
704
    else:
705
        return "method"
706
```