0
# LibCST
1

2
A concrete syntax tree parser and serializer library for Python that parses Python 3.0-3.13 source code as a CST tree that preserves all formatting details including comments, whitespaces, and parentheses. LibCST creates a compromise between Abstract Syntax Trees (AST) and traditional Concrete Syntax Trees (CST) by carefully reorganizing node types to create a lossless CST that looks and feels like an AST, making it ideal for building automated refactoring applications and linters.
3

4
## Package Information
5

6
- **Package Name**: libcst
7
- **Language**: Python
8
- **Installation**: `pip install libcst`
9

10
## Core Imports
11

12
```python
13
import libcst as cst
14
```
15

16
Common parsing functions:
17

18
```python
19
from libcst import parse_module, parse_statement, parse_expression
20
```
21

22
Visitor framework:
23

24
```python
25
from libcst import CSTVisitor, CSTTransformer, MetadataDependent
26
```
27

28
## Basic Usage
29

30
```python
31
import libcst as cst
32

33
# Parse Python code into a CST
34
source_code = '''
35
def hello_world():
36
    print("Hello, World!")
37
    return 42
38
'''
39

40
# Parse module preserving all formatting
41
module = cst.parse_module(source_code)
42

43
# Transform code using a visitor
44
class NameCollector(cst.CSTVisitor):
45
    def __init__(self):
46
        self.names = []
47
    
48
    def visit_Name(self, node):
49
        self.names.append(node.value)
50

51
visitor = NameCollector()
52
module.visit(visitor)
53
print(visitor.names)  # ['hello_world', 'print']
54

55
# Generate code back from CST (lossless)
56
generated_code = module.code
57
print(generated_code)  # Identical to original including whitespace
58
```
59

60
## Architecture
61

62
LibCST's design is built around several key components:
63

64
- **CST Nodes**: Immutable dataclasses representing every element of Python syntax
65
- **Visitor Framework**: Pattern for traversing and transforming CST trees
66
- **Metadata Providers**: System for attaching semantic information to nodes
67
- **Matcher System**: Declarative pattern matching for finding specific code patterns
68
- **Codemod Framework**: High-level tools for building code transformation applications
69

70
This architecture enables LibCST to serve as the foundation for sophisticated code analysis and transformation tools while maintaining perfect source code fidelity.
71

72
## Capabilities
73

74
### Core Parsing Functions
75

76
Parse Python source code into concrete syntax trees that preserve all formatting details including comments, whitespace, and parentheses.
77

78
```python { .api }
79
def parse_module(source: Union[str, bytes], config: Optional[PartialParserConfig] = None) -> Module: ...
80
def parse_statement(source: str, config: Optional[PartialParserConfig] = None) -> Union[SimpleStatementLine, BaseCompoundStatement]: ...
81
def parse_expression(source: str, config: Optional[PartialParserConfig] = None) -> BaseExpression: ...
82
```
83

84
[Parsing](./parsing.md)
85

86
### Visitor Framework
87

88
Traverse and transform CST trees using the visitor pattern, with support for metadata-dependent analysis and batched processing.
89

90
```python { .api }
91
class CSTVisitor:
92
    def visit(self, node: CSTNode) -> None: ...
93
    def leave(self, node: CSTNode) -> None: ...
94

95
class CSTTransformer:
96
    def visit(self, node: CSTNode) -> None: ...
97
    def leave(self, original_node: CSTNode, updated_node: CSTNode) -> CSTNode: ...
98

99
class MetadataDependent:
100
    METADATA_DEPENDENCIES: Sequence[ProviderT] = ()
101
    def resolve(self, node: CSTNode) -> Any: ...
102
```
103

104
[Visitor Framework](./visitors.md)
105

106
### Pattern Matching
107

108
Declarative pattern matching system for finding, extracting, and replacing specific code patterns in CST trees.
109

110
```python { .api }
111
def matches(node: CSTNode, matcher: Union[CSTNode, MatcherPattern]) -> bool: ...
112
def findall(tree: CSTNode, matcher: Union[CSTNode, MatcherPattern]) -> Sequence[CSTNode]: ...
113
def extract(node: CSTNode, matcher: Union[CSTNode, MatcherPattern]) -> Dict[str, Union[CSTNode, Sequence[CSTNode]]]: ...
114
def replace(tree: CSTNode, matcher: Union[CSTNode, MatcherPattern], replacement: Union[CSTNode, Callable]) -> CSTNode: ...
115
```
116

117
[Pattern Matching](./matchers.md)
118

119
### Metadata Analysis
120

121
Attach semantic information to CST nodes including scope analysis, position tracking, qualified names, and type inference.
122

123
```python { .api }
124
class MetadataWrapper:
125
    def __init__(self, module: Module, cache: Optional[Dict] = None) -> None: ...
126
    def resolve(self, provider: Type[BaseMetadataProvider]) -> Mapping[CSTNode, Any]: ...
127
    def resolve_many(self, providers: Sequence[Type[BaseMetadataProvider]]) -> Dict[Type[BaseMetadataProvider], Mapping[CSTNode, Any]]: ...
128

129
class ScopeProvider(BaseMetadataProvider):
130
    def visit_Module(self, node: Module) -> None: ...
131

132
class PositionProvider(BaseMetadataProvider):
133
    def visit_Module(self, node: Module) -> None: ...
134
```
135

136
[Metadata Analysis](./metadata.md)
137

138
### Codemod Framework
139

140
High-level framework for building automated code transformation applications with context management, parallel processing, and testing utilities.
141

142
```python { .api }
143
class CodemodCommand:
144
    def transform_module(self, tree: Module) -> Module: ...
145

146
class VisitorBasedCodemodCommand(CodemodCommand):
147
    def leave_Module(self, original_node: Module, updated_node: Module) -> Module: ...
148

149
def transform_module(code: str, command: CodemodCommand) -> TransformResult: ...
150
def parallel_exec_transform_with_prettyprint(command: CodemodCommand, files: Sequence[str]) -> ParallelTransformResult: ...
151
```
152

153
[Codemod Framework](./codemods.md)
154

155
### CST Node Types
156

157
Complete set of immutable dataclasses representing every element of Python syntax from expressions and statements to operators and whitespace.
158

159
```python { .api }
160
class Module(CSTNode):
161
    body: Sequence[Union[SimpleStatementLine, BaseCompoundStatement]]
162
    header: Sequence[Union[SimpleStatementLine, BaseCompoundStatement]]
163
    footer: Sequence[Union[SimpleStatementLine, BaseCompoundStatement]]
164

165
class Name(BaseExpression):
166
    value: str
167
    
168
class FunctionDef(BaseCompoundStatement):
169
    name: Name
170
    params: Parameters
171
    body: BaseSuite
172
    decorators: Sequence[Decorator]
173
```
174

175
[CST Nodes](./nodes.md)
176

177
### Command-Line Tool
178

179
Comprehensive command-line interface for LibCST operations including parsing, visualization, and code transformation execution.
180

181
```python { .api }
182
# Main CLI commands
183
python -m libcst.tool print <file>     # Display CST tree representation
184
python -m libcst.tool codemod <command> <files>  # Execute code transformations
185
python -m libcst.tool list             # Show available codemod commands
186
python -m libcst.tool initialize       # Create configuration files
187
```
188

189
[CLI Tool Reference](./cli-tool.md)
190

191
### Utilities and Helpers
192

193
Helper functions, debugging utilities, and testing framework for LibCST development and programmatic usage.
194

195
```python { .api }
196
# Helper functions
197
from libcst.helpers import calculate_module_and_package, parse_template_module, get_full_name_for_node
198

199
# Display utilities  
200
from libcst.display import dump, dump_graphviz
201

202
# Testing framework
203
from libcst.testing import UnitTest, CodemodTest
204
```
205

206
[Utilities and Helpers](./utilities.md)
207

208
## Types
209

210
```python { .api }
211
# Base node types
212
class CSTNode:
213
    def visit(self, visitor: CSTVisitor) -> CSTNode: ...
214
    def with_changes(self, **changes: Any) -> CSTNode: ...
215

216
# Core expression and statement base classes  
217
class BaseExpression(CSTNode): ...
218
class BaseStatement(CSTNode): ...
219
class BaseCompoundStatement(BaseStatement): ...
220
class BaseSmallStatement(BaseStatement): ...
221

222
# Visitor types
223
CSTNodeT = TypeVar("CSTNodeT", bound=CSTNode)
224
CSTVisitorT = TypeVar("CSTVisitorT", bound="CSTVisitor")
225

226
# Exception types
227
class CSTValidationError(Exception): ...
228
class ParserSyntaxError(Exception): ...
229
class CSTLogicError(Exception): ...
230
class MetadataException(Exception): ...
231

232
# Transform results
233
class TransformResult:
234
    code: Optional[str]
235
    encoding: str
236
    
237
class TransformSuccess(TransformResult): ...
238
class TransformFailure(TransformResult): 
239
    error: Exception
240
```