0
# Checker System
1

2
Advanced code analysis engine that performs AST-based static analysis with scope tracking and binding management. The Checker class provides fine-grained control over the analysis process and access to detailed results, supporting custom analysis workflows and integration with development tools.
3

4
The Checker class is the core of pyflakes' analysis engine, implementing a comprehensive visitor pattern for Python AST nodes with sophisticated scope and binding management.
5

6
## Capabilities
7

8
### Core Checker Class
9

10
Main analysis engine that walks Python AST nodes and maintains scope information to detect various code issues.
11

12
```python { .api }
13
class Checker:
14
    """I check the cleanliness and sanity of Python code."""
15
    
16
    def __init__(self, tree, filename='(none)', builtins=None, withDoctest='PYFLAKES_DOCTEST' in os.environ, file_tokens=()):
17
        """
18
        Initialize the checker with an AST tree.
19

20
        Parameters:
21
        - tree: AST tree from ast.parse()
22
        - filename (str): Name of the file being checked (for error reporting)
23
        - builtins (set, optional): Additional builtin names to recognize (extends default set)
24
        - withDoctest (bool): Whether to check doctest code blocks (controlled by PYFLAKES_DOCTEST env var)
25
        - file_tokens (tuple): Deprecated parameter for token information (warns when used)
26
        """
27
    
28
    # Core attributes
29
    messages: list          # List of Message objects representing found issues
30
    filename: str           # Filename being checked
31
    deadScopes: list        # List of completed scopes for analysis
32
    scopeStack: list        # Stack of active scopes during traversal
33
    exceptHandlers: list    # Stack of exception handler names for context
34
    root                    # Root AST node
35
    nodeDepth: int          # Current depth in AST traversal
36
    offset                  # Line/column offset for node adjustment
37
    withDoctest: bool       # Whether doctest processing is enabled
38
    
39
    # Properties
40
    @property
41
    def futuresAllowed(self) -> bool:
42
        """Whether __future__ imports are allowed at current position."""
43
    
44
    @property
45
    def annotationsFutureEnabled(self) -> bool:
46
        """Whether 'from __future__ import annotations' is active."""
47
    
48
    @property
49
    def scope(self):
50
        """Current scope (last item in scopeStack)."""
51
    
52
    @property
53
    def _in_postponed_annotation(self) -> bool:
54
        """Whether currently in a postponed annotation context."""
55
```
56

57
**Usage:**
58

59
```python
60
import ast
61
import pyflakes.checker
62

63
# Parse Python code into AST
64
code = """
65
import os
66
unused_var = 42
67
print(undefined_var)
68
"""
69

70
tree = ast.parse(code, filename='example.py')
71
checker = pyflakes.checker.Checker(tree, filename='example.py')
72

73
# Access results
74
print(f"Found {len(checker.messages)} issues:")
75
for message in checker.messages:
76
    print(f"  {message}")
77

78
# Access scope information
79
print(f"Analyzed {len(checker.deadScopes)} scopes")
80
```
81

82
### Core Analysis Methods
83

84
Primary methods for managing the analysis process and reporting issues.
85

86
```python { .api }
87
def report(self, messageClass, *args, **kwargs):
88
    """Add a message to the results."""
89

90
def deferFunction(self, callable):
91
    """Schedule a function handler to be called just before completion."""
92

93
def checkDeadScopes(self):
94
    """Analyze completed scopes for unused names."""
95

96
def handleNode(self, node, parent):
97
    """Process an AST node during analysis."""
98

99
def handleChildren(self, tree, omit=None):
100
    """Process child nodes with optional omissions."""
101

102
def getNodeHandler(self, node_class):
103
    """Get handler method for AST node type."""
104

105
def addBinding(self, node, value):
106
    """Add name binding to current scope."""
107
```
108

109
### Scope Management Methods
110

111
Methods for managing Python scopes and their relationships.
112

113
```python { .api }
114
def in_scope(self, cls):
115
    """Context manager for scope management."""
116

117
def getScopeNode(self, node):
118
    """Find scope-defining ancestor."""
119

120
def getParent(self, node):
121
    """Get meaningful parent of a node."""
122

123
def getCommonAncestor(self, lnode, rnode, stop):
124
    """Find common ancestor of two nodes."""
125

126
def descendantOf(self, node, ancestors, stop):
127
    """Check if node is descendant of ancestors."""
128

129
def differentForks(self, lnode, rnode):
130
    """Check if nodes are on different conditional branches."""
131
```
132

133
### Name Binding Methods
134

135
Methods for handling variable assignments and name resolution.
136

137
```python { .api }
138
def handleNodeLoad(self, node, parent):
139
    """Handle name loading/usage."""
140

141
def handleNodeStore(self, node):
142
    """Handle name storage/assignment."""
143

144
def handleNodeDelete(self, node):
145
    """Handle name deletion."""
146
```
147

148
### Annotation Processing Methods
149

150
Methods for handling type annotations and forward references.
151

152
```python { .api }
153
def handleAnnotation(self, annotation, node):
154
    """Process type annotations."""
155

156
def handleStringAnnotation(self, s, node, ref_lineno, ref_col_offset, err):
157
    """Process string annotations."""
158

159
def handle_annotation_always_deferred(self, annotation, parent):
160
    """Always defer annotation processing."""
161

162
def _enter_annotation(self, ann_type=AnnotationState.BARE):
163
    """Context manager for annotations."""
164
```
165

166
### AST Node Handlers
167

168
The Checker class implements handlers for all Python AST node types. Each handler is named after the AST node type in uppercase.
169

170
#### Statement Handlers
171

172
```python { .api }
173
def FUNCTIONDEF(self, node):
174
    """Handle function definitions."""
175

176
def ASYNCFUNCTIONDEF(self, node):
177
    """Handle async function definitions."""
178

179
def CLASSDEF(self, node):
180
    """Handle class definitions."""
181

182
def RETURN(self, node):
183
    """Handle return statements with scope validation."""
184

185
def YIELD(self, node):
186
    """Handle yield expressions with scope validation."""
187

188
def GLOBAL(self, node):
189
    """Handle global declarations."""
190

191
def IMPORT(self, node):
192
    """Handle import statements."""
193

194
def IMPORTFROM(self, node):
195
    """Handle 'from X import Y' statements."""
196

197
def ASSIGN(self, node):
198
    """Handle assignments."""
199

200
def AUGASSIGN(self, node):
201
    """Handle augmented assignments (+=, etc.)."""
202

203
def ANNASSIGN(self, node):
204
    """Handle annotated assignments."""
205

206
def FOR(self, node):
207
    """Handle for loops."""
208

209
def ASYNCFOR(self, node):
210
    """Handle async for loops."""
211

212
def WHILE(self, node):
213
    """Handle while loops."""
214

215
def IF(self, node):
216
    """Handle if statements with tuple condition checking."""
217

218
def WITH(self, node):
219
    """Handle with statements."""
220

221
def ASYNCWITH(self, node):
222
    """Handle async with statements."""
223

224
def TRY(self, node):
225
    """Handle try/except blocks with handler tracking."""
226

227
def EXCEPTHANDLER(self, node):
228
    """Handle exception handlers."""
229

230
def RAISE(self, node):
231
    """Handle raise statements with NotImplemented checking."""
232

233
def ASSERT(self, node):
234
    """Handle assert statements with tuple condition checking."""
235

236
def CONTINUE(self, node):
237
    """Handle continue/break statements with context validation."""
238

239
def BREAK(self, node):
240
    """Handle break statements."""
241

242
def PASS(self, node):
243
    """Handle pass statements."""
244
```
245

246
#### Expression Handlers
247

248
```python { .api }
249
def NAME(self, node):
250
    """Handle name references (load/store/delete)."""
251

252
def CALL(self, node):
253
    """Handle function calls with format string validation."""
254

255
def SUBSCRIPT(self, node):
256
    """Handle subscript expressions with special typing logic."""
257

258
def ATTRIBUTE(self, node):
259
    """Handle attribute access."""
260

261
def BINOP(self, node):
262
    """Handle binary operations with percent format checking."""
263

264
def COMPARE(self, node):
265
    """Handle comparison operations with literal checking."""
266

267
def TUPLE(self, node):
268
    """Handle tuples with starred expression validation."""
269

270
def LIST(self, node):
271
    """Handle lists."""
272

273
def DICT(self, node):
274
    """Handle dictionaries with duplicate key detection."""
275

276
def SET(self, node):
277
    """Handle sets."""
278

279
def CONSTANT(self, node):
280
    """Handle constants with string annotation support."""
281

282
def JOINEDSTR(self, node):
283
    """Handle f-strings with placeholder validation."""
284

285
def TEMPLATESTR(self, node):
286
    """Handle template strings (t-strings)."""
287

288
def LAMBDA(self, node):
289
    """Handle lambda expressions and function arguments."""
290

291
def GENERATOREXP(self, node):
292
    """Handle generator expressions and comprehensions."""
293

294
def LISTCOMP(self, node):
295
    """Handle list comprehensions."""
296

297
def DICTCOMP(self, node):
298
    """Handle dictionary comprehensions."""
299

300
def SETCOMP(self, node):
301
    """Handle set comprehensions."""
302
```
303

304
#### Type System Handlers (Python 3.12+)
305

306
```python { .api }
307
def TYPEVAR(self, node):
308
    """Handle type variable definitions."""
309

310
def TYPEALIAS(self, node):
311
    """Handle type alias definitions."""
312

313
def PARAMSPEC(self, node):
314
    """Handle parameter specifications."""
315

316
def TYPEVARTUPLE(self, node):
317
    """Handle type variable tuples."""
318
```
319

320
#### Pattern Matching Handlers (Python 3.10+)
321

322
```python { .api }
323
def MATCH(self, node):
324
    """Handle match statements."""
325

326
def MATCH_CASE(self, node):
327
    """Handle match cases."""
328

329
def MATCHAS(self, node):
330
    """Handle match as patterns."""
331

332
def MATCHCLASS(self, node):
333
    """Handle match class patterns."""
334

335
def MATCHMAPPING(self, node):
336
    """Handle match mapping patterns."""
337

338
def MATCHOR(self, node):
339
    """Handle match or patterns."""
340

341
def MATCHSEQUENCE(self, node):
342
    """Handle match sequence patterns."""
343

344
def MATCHSINGLETON(self, node):
345
    """Handle match singleton patterns."""
346

347
def MATCHSTAR(self, node):
348
    """Handle match star patterns."""
349

350
def MATCHVALUE(self, node):
351
    """Handle match value patterns."""
352
```
353

354
### Utility Methods
355

356
Helper methods for common analysis tasks.
357

358
```python { .api }
359
def isLiteralTupleUnpacking(self, node) -> bool:
360
    """Check for literal tuple unpacking."""
361

362
def isDocstring(self, node) -> bool:
363
    """Check if node is a docstring."""
364

365
def getDocstring(self, node):
366
    """Extract docstring content and line number."""
367

368
def handleDoctests(self, node):
369
    """Process doctest examples in docstrings."""
370

371
def ignore(self, node):
372
    """No-op handler for ignored node types."""
373

374
def _in_doctest(self) -> bool:
375
    """Check if currently in doctest scope."""
376
```
377

378
### Format String Validation
379

380
Methods for validating string formatting operations.
381

382
```python { .api }
383
def _handle_string_dot_format(self, node):
384
    """Validate .format() method calls."""
385

386
def _handle_percent_format(self, node):
387
    """Validate % format operations."""
388
```
389

390
## Internal Systems
391

392
### Deferred Analysis System
393

394
The checker implements a deferred analysis system to handle forward references and ensure proper name resolution order.
395

396
```python
397
# Deferred functions are stored and executed after initial traversal
398
checker._deferred = []  # Queue of deferred function handlers
399
checker._run_deferred()  # Execute all deferred functions
400
```
401

402
**Key Features:**
403
- Functions bodies are analyzed after global scope is complete
404
- Ensures all global names are visible to function analysis
405
- Preserves scope context when handlers are eventually executed
406
- Critical for handling forward references and decorators
407

408
### Binding Management System
409

410
Comprehensive system for tracking name bindings across all scopes using a hierarchical class system.
411

412
#### Binding Class Hierarchy
413

414
```python { .api }
415
class Binding:
416
    """Base class for all name bindings."""
417
    
418
    def __init__(self, name, source):
419
        """
420
        Parameters:
421
        - name (str): The bound name
422
        - source: AST node where binding occurs
423
        """
424
    
425
    name: str           # The bound name
426
    source             # AST node where binding occurs  
427
    used               # Usage tracking: False or (scope, node) tuple
428
    
429
    def redefines(self, other) -> bool:
430
        """Determines if this binding redefines another."""
431

432
class Definition(Binding):
433
    """Base class for bindings that define functions or classes."""
434
    
435
    def redefines(self, other) -> bool:
436
        """Can redefine other definitions or assignments."""
437

438
class Assignment(Binding):
439
    """Regular variable assignments (x = value)."""
440

441
class NamedExprAssignment(Assignment):
442
    """Walrus operator assignments (x := value)."""
443

444
class Annotation(Binding):
445
    """Type annotations without values (x: int)."""
446
    
447
    def redefines(self, other) -> bool:
448
        """Annotations don't redefine names."""
449

450
class Argument(Binding):
451
    """Function parameters."""
452

453
class FunctionDefinition(Definition):
454
    """Function definitions (def func():)."""
455

456
class ClassDefinition(Definition):
457
    """Class definitions (class MyClass:)."""
458

459
class Builtin(Definition):
460
    """Built-in names (like print, len)."""
461
```
462

463
#### Import-Related Bindings
464

465
```python { .api }
466
class Importation(Definition):
467
    """Standard import statements (import module)."""
468
    
469
    def __init__(self, name, source, full_name=None):
470
        """
471
        Parameters:
472
        - name (str): Local name for the import
473
        - source: AST node
474
        - full_name (str): Complete import path
475
        """
476
    
477
    fullName: str       # Complete import path
478
    redefined: list     # List of redefinition nodes
479
    
480
    def _has_alias(self) -> bool:
481
        """Whether import uses 'as' clause."""
482
    
483
    @property
484
    def source_statement(self) -> str:
485
        """Reconstructs the original import statement."""
486

487
class SubmoduleImportation(Importation):
488
    """Submodule imports (import package.module)."""
489

490
class ImportationFrom(Importation):
491
    """From imports (from module import name)."""
492

493
class StarImportation(Importation):
494
    """Star imports (from module import *)."""
495

496
class FutureImportation(ImportationFrom):
497
    """Future imports (from __future__ import feature)."""
498

499
class ExportBinding(Binding):
500
    """__all__ assignments for module exports."""
501
    
502
    def __init__(self, name, source, scope):
503
        """Parses __all__ assignments and concatenations."""
504
    
505
    names: list         # List of exported names from __all__
506
```
507

508
#### Binding Creation Process
509

510
Bindings are created through several key methods:
511

512
**Name Storage (`handleNodeStore`):**
513
- Creates `Annotation` for type-only annotations (`x: int`)
514
- Creates `Binding` for loop variables and tuple unpacking
515
- Creates `ExportBinding` for `__all__` assignments
516
- Creates `NamedExprAssignment` for walrus operators (`x := value`)
517
- Creates `Assignment` for regular assignments (`x = value`)
518

519
**Import Processing:**
520
- `IMPORT` method creates `Importation` or `SubmoduleImportation`
521
- `IMPORTFROM` method creates `ImportationFrom`, `StarImportation`, or `FutureImportation`
522

523
**Definition Processing:**
524
- `FUNCTIONDEF` creates `FunctionDefinition` bindings
525
- `CLASSDEF` creates `ClassDefinition` bindings
526
- `ARG` creates `Argument` bindings for function parameters
527

528
#### Usage Tracking
529

530
Bindings track their usage through the `used` attribute:
531
- `False` - Never used
532
- `(scope, node)` tuple - Used, with reference to usage location
533

534
Usage is recorded in `handleNodeLoad(node, parent)` when names are accessed.
535

536
### Scope System
537

538
Manages nested Python scopes with their specific rules and behaviors using a hierarchy of scope classes.
539

540
#### Scope Class Hierarchy
541

542
```python { .api }
543
class Scope(dict):
544
    """Base scope class - dictionary of name -> binding."""
545
    
546
    def __init__(self, filename=None):
547
        """Initialize scope with optional filename."""
548
    
549
    def __contains__(self, key):
550
        """Check if name is bound in this scope."""
551
    
552
    def unusedAssignments(self):
553
        """Return unused assignments in this scope."""
554

555
class ModuleScope(Scope):
556
    """Module-level scope."""
557
    
558
    def __init__(self, filename=None):
559
        self.importStarred = False  # Whether any star imports exist
560
        self.futureAnnotations = False  # Whether __future__ annotations enabled
561

562
class ClassScope(Scope):
563
    """Class definition scope."""
564
    
565
    def __init__(self, filename=None):
566
        # Classes don't participate in normal name resolution
567
        pass
568

569
class FunctionScope(Scope):
570
    """Function and method scope."""
571
    
572
    def __init__(self, filename=None):
573
        self.globals = {}           # Global declarations
574
        self.nonlocals = {}         # Nonlocal declarations
575
        self.indirectGlobals = {}   # Names that may become global
576
        self.indirectNonlocals = {}  # Names that may become nonlocal
577
    
578
    def unused_assignments(self):
579
        """Return unused assignments in function scope."""
580
    
581
    def unused_annotations(self):
582
        """Return unused type annotations in function scope."""
583

584
class GeneratorScope(Scope):
585
    """Generator expressions and comprehensions."""
586
    
587
    def __init__(self, filename=None):
588
        # Generator scopes can access class scope variables
589
        pass
590

591
class TypeScope(Scope):
592
    """Type parameter scope (Python 3.12+)."""
593

594
class DoctestScope(ModuleScope):
595
    """Doctest execution scope."""
596
```
597

598
#### Scope Stack Management
599

600
The checker maintains a stack of active scopes during AST traversal:
601

602
```python { .api }
603
# Core scope management attributes
604
scopeStack: list        # Stack of active scopes during traversal
605
deadScopes: list        # List of completed scopes for analysis
606

607
@property
608
def scope(self):
609
    """Current scope (last item in scopeStack)."""
610
    return self.scopeStack[-1] if self.scopeStack else None
611

612
def in_scope(self, cls):
613
    """Context manager for scope management."""
614
    # Pushes new scope, yields it, then pops and moves to deadScopes
615
```
616

617
#### Scope Resolution Rules
618

619
**Name Lookup Order:**
620
1. Current scope (innermost)
621
2. Enclosing function scopes (if any)
622
3. Global scope (module level)
623
4. Built-in scope
624

625
**Special Scoping Rules:**
626
- **Class scopes** don't participate in normal name resolution for most names
627
- **Generator scopes** can access class variables directly
628
- **Global/nonlocal** declarations affect where names are stored
629
- **Star imports** set `scope.importStarred = True` to disable undefined name warnings
630

631
#### Scope-Binding Integration
632

633
Scopes store bindings and provide methods for analysis:
634

635
```python
636
# Adding bindings to scopes
637
def addBinding(self, node, value):
638
    """Add name binding to appropriate scope."""
639
    # Finds correct scope in stack based on global/nonlocal declarations
640
    # Handles redefinition checking and reporting
641
    # Updates binding usage information
642

643
# Scope analysis after traversal
644
def checkDeadScopes(self):
645
    """Analyze completed scopes for unused names."""
646
    # Reports unused imports, variables, and annotations
647
    # Validates __all__ exports
648
    # Processes star import usage patterns
649
```
650

651
#### Advanced Scope Features
652

653
**Module Scope Features:**
654
- Tracks future imports (`from __future__ import annotations`)
655
- Manages star imports and their effects on undefined name detection
656
- Handles `__all__` export validation
657

658
**Function Scope Features:**
659
- Tracks global and nonlocal declarations
660
- Identifies unused assignments and annotations
661
- Manages function argument bindings
662
- Handles closure variable access
663

664
**Class Scope Features:**
665
- Class variables are not visible in methods (normal Python scoping)
666
- Class scope bindings tracked but don't affect name resolution
667
- Special handling for class decorators and base classes
668

669
**Generator Scope Features:**
670
- Comprehensions create isolated scopes
671
- Can access enclosing class variables (unlike functions)
672
- Iterator variables don't leak to enclosing scope
673

674
### Annotation Processing System
675

676
Handles Python type annotations with support for forward references and `from __future__ import annotations`.
677

678
**Annotation States:**
679
- `BARE` - Direct annotation context
680
- `STRINGIZED` - String annotation context
681
- `POSTPONED` - Deferred annotation context
682

683
**Features:**
684
- String annotation parsing with error handling
685
- Support for `typing` module constructs
686
- Forward reference resolution
687
- Future annotations behavior
688

689
### Message Reporting System
690

691
Structured system for collecting and reporting code issues.
692

693
**Features:**
694
- All messages stored in `messages` list
695
- Detailed location information (file, line, column)
696
- Message classification by severity and type
697
- Integration with reporter system for output formatting
698

699
## Advanced Usage Examples
700

701
### Custom Analysis with Deferred Functions
702

703
```python
704
import ast
705
import pyflakes.checker
706

707
def custom_analysis_callback():
708
    """Custom analysis to run after main traversal."""
709
    print("Running custom analysis...")
710
    # Access checker state here
711
    return
712

713
code = """
714
def func():
715
    global_var = 42  # This will be analyzed after global scope
716
    
717
global_var = 10
718
"""
719

720
tree = ast.parse(code, 'test.py')
721
checker = pyflakes.checker.Checker(tree, 'test.py')
722

723
# Add custom deferred analysis
724
checker.deferFunction(custom_analysis_callback)
725

726
# Analysis completes with our custom function
727
print(f"Found {len(checker.messages)} issues")
728
```
729

730
### Scope Analysis
731

732
```python
733
import ast
734
import pyflakes.checker
735

736
code = """
737
class MyClass:
738
    x = 1
739
    
740
    def method(self):
741
        y = 2
742
        
743
        def inner():
744
            z = 3
745
"""
746

747
tree = ast.parse(code, 'test.py')
748
checker = pyflakes.checker.Checker(tree, 'test.py')
749

750
# Analyze completed scopes
751
print(f"Analyzed {len(checker.deadScopes)} scopes:")
752
for i, scope in enumerate(checker.deadScopes):
753
    print(f"  Scope {i}: {type(scope).__name__}")
754
    print(f"    Bindings: {list(scope.keys())}")
755
```
756

757
### Message Analysis by Type
758

759
```python
760
import ast
761
import pyflakes.checker
762
from pyflakes.messages import UnusedImport, UndefinedName
763

764
code = """
765
import os
766
import sys
767
print(undefined_name)
768
unused_var = 42
769
"""
770

771
tree = ast.parse(code, 'test.py')
772
checker = pyflakes.checker.Checker(tree, 'test.py')
773

774
# Categorize messages
775
unused_imports = [m for m in checker.messages if isinstance(m, UnusedImport)]
776
undefined_names = [m for m in checker.messages if isinstance(m, UndefinedName)]
777

778
print(f"Unused imports: {len(unused_imports)}")
779
print(f"Undefined names: {len(undefined_names)}")
780

781
for msg in checker.messages:
782
    print(f"{type(msg).__name__}: {msg}")
783
```
784

785
### Annotation System Usage
786

787
The checker provides sophisticated annotation handling that can be extended:
788

789
```python
790
import ast
791
import pyflakes.checker
792

793
code = '''
794
from typing import List, Dict
795
from __future__ import annotations
796

797
def func(x: List[int]) -> Dict[str, int]:
798
    """Function with type annotations."""
799
    return {"key": x[0]}
800

801
# Forward reference
802
def forward_ref() -> SomeClass:
803
    pass
804

805
class SomeClass:
806
    pass
807
'''
808

809
tree = ast.parse(code, 'test.py')
810
checker = pyflakes.checker.Checker(tree, 'test.py')
811

812
# Check if future annotations are enabled
813
print(f"Future annotations enabled: {checker.annotationsFutureEnabled}")
814

815
# All annotations are processed appropriately
816
print(f"Analysis complete with {len(checker.messages)} issues")
817
```