0
# Parso
1

2
A Python parser that supports error recovery and round-trip parsing for different Python versions. Parso enables parsing and analysis of Python code with the ability to handle syntactically incorrect code and maintain precise position information for all tokens. Originally part of the Jedi project, it provides a small but comprehensive API for parsing Python code and analyzing syntax trees.
3

4
## Package Information
5

6
- **Package Name**: parso
7
- **Language**: Python
8
- **Installation**: `pip install parso`
9

10
## Core Imports
11

12
```python
13
import parso
14
```
15

16
For working with specific components:
17

18
```python
19
from parso import parse, load_grammar, Grammar, ParserSyntaxError
20
from parso import split_lines, python_bytes_to_unicode
21
from parso.tree import NodeOrLeaf, BaseNode, Leaf
22
from parso.python.tree import Module, Function, Class, Name
23
```
24

25
## Basic Usage
26

27
```python
28
import parso
29

30
# Parse Python code - simplest approach
31
module = parso.parse('def hello(): return "world"')
32
print(module.children[0])  # Function definition node
33

34
# Parse with specific Python version
35
module = parso.parse('x := 42', version="3.8")  # Walrus operator
36
expr = module.children[0].children[0]
37
print(expr.get_code())  # 'x := 42'
38

39
# Load grammar for advanced usage
40
grammar = parso.load_grammar(version="3.9")
41
module = grammar.parse('hello + 1')
42

43
# Navigate the syntax tree
44
expr = module.children[0]  # Expression statement
45
name = expr.children[0].children[0]  # 'hello' name
46
print(name.value)  # 'hello'
47
print(name.start_pos)  # (1, 0)
48
print(name.end_pos)    # (1, 5)
49

50
# Handle syntax errors with error recovery
51
grammar = parso.load_grammar()
52
module = grammar.parse('def broken(: pass')  # Invalid syntax
53
errors = list(grammar.iter_errors(module))
54
for error in errors:
55
    print(f"Error: {error.message}")
56
```
57

58
## Architecture
59

60
Parso uses a multi-layered architecture for robust Python parsing:
61

62
- **Grammar System**: Version-specific Python grammars (3.6-3.14) with BNF-based parser generation
63
- **Tokenizer**: Python-aware tokenization supporting f-strings, encoding detection, and prefix handling
64
- **Parser**: LR-style parser with error recovery capabilities for handling incomplete/invalid code
65
- **Syntax Tree**: Rich AST-like tree structure with precise position tracking and code regeneration
66
- **Normalization**: PEP 8 formatting and error detection system for code quality analysis
67

68
This design enables parso to serve as the foundation for IDEs, linters, code formatters, and other Python code analysis tools that need to work with both valid and invalid Python code.
69

70
## Capabilities
71

72
### Core Parsing Functions
73

74
Main entry points for parsing Python code, including version-specific grammar loading and high-level parsing utilities that handle the most common use cases.
75

76
```python { .api }
77
def parse(code=None, **kwargs): ...
78
def load_grammar(*, version=None, path=None): ...
79
```
80

81
[Core Parsing](./core-parsing.md)
82

83
### Grammar System
84

85
Grammar classes that handle parsing with different Python versions, error recovery, caching, and advanced parsing options for production use.
86

87
```python { .api }
88
class Grammar:
89
    def parse(self, code=None, **kwargs): ...
90
    def iter_errors(self, node): ...
91
    def refactor(self, base_node, node_to_str_map): ...
92

93
class PythonGrammar(Grammar): ...
94
```
95

96
[Grammar System](./grammar-system.md)
97

98
### Syntax Tree Navigation
99

100
Base classes and methods for navigating and manipulating the parsed syntax tree, including position tracking, sibling navigation, and code regeneration.
101

102
```python { .api }
103
class NodeOrLeaf:
104
    def get_root_node(self): ...
105
    def get_next_sibling(self): ...
106
    def get_previous_sibling(self): ...
107
    def get_code(self, include_prefix=True): ...
108
    def dump(self, *, indent=4): ...
109

110
class BaseNode(NodeOrLeaf): ...
111
class Leaf(NodeOrLeaf): ...
112
```
113

114
[Tree Navigation](./tree-navigation.md)
115

116
### Python Syntax Elements
117

118
Python-specific node types representing functions, classes, imports, control flow, and all Python language constructs with specialized methods for analysis.
119

120
```python { .api }
121
class Module(Scope):
122
    def get_used_names(self): ...
123
    def iter_imports(self): ...
124

125
class Function(ClassOrFunc):
126
    def get_params(self): ...
127
    def iter_yield_exprs(self): ...
128
    def is_generator(self): ...
129

130
class Class(ClassOrFunc):
131
    def get_super_arglist(self): ...
132

133
class Name(PythonLeaf):
134
    def is_definition(self, include_setitem=False): ...
135
    def get_definition(self, import_name_always=False, include_setitem=False): ...
136
```
137

138
[Python Elements](./python-elements.md)
139

140
### Tokenization
141

142
Low-level tokenization functions and classes for converting Python source code into tokens, handling encoding, f-strings, and Python version differences.
143

144
```python { .api }
145
def tokenize(code, version_info, start_pos=(1, 0)): ...
146
def tokenize_lines(lines, version_info, start_pos=(1, 0)): ...
147

148
class PythonTokenTypes: ...
149
```
150

151
[Tokenization](./tokenization.md)
152

153
### Error Handling and Analysis
154

155
Error detection, syntax error reporting, and code quality analysis including PEP 8 normalization and custom rule systems.
156

157
```python { .api }
158
class ParserSyntaxError(Exception): ...
159
class ErrorFinder: ...
160
class Normalizer: ...
161
class PEP8Normalizer(Normalizer): ...
162
```
163

164
[Error Handling](./error-handling.md)
165

166
### Utilities and Helpers
167

168
Utility functions for text processing, version handling, encoding detection, and file I/O operations that support the parsing infrastructure.
169

170
```python { .api }
171
def split_lines(string, keepends=False): ...
172
def python_bytes_to_unicode(source, encoding='utf-8', errors='strict'): ...
173
def parse_version_string(version=None): ...
174
def version_info(): ...
175

176
class FileIO: ...
177
class PythonVersionInfo: ...
178
```
179

180
[Utilities](./utilities.md)
181

182
## Types
183

184
### Core Types
185

186
```python { .api }
187
class PythonVersionInfo:
188
    major: int
189
    minor: int
190

191
class Version:
192
    major: int
193
    minor: int
194
    micro: int
195
```
196

197
### Error Types
198

199
```python { .api }
200
class ParserSyntaxError(Exception):
201
    message: str
202
    error_leaf: ErrorLeaf
203

204
class InternalParseError(Exception):
205
    msg: str
206
    type: Any
207
    value: str
208
    start_pos: tuple[int, int]
209
```