CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-parso

A Python parser that supports error recovery and round-trip parsing for different Python versions

Pending
Overview
Eval results
Files

core-parsing.mddocs/

Core Parsing Functions

Main entry points for parsing Python code. These functions provide the simplest interface to parso's parsing capabilities and handle the most common use cases without requiring detailed knowledge of the grammar system.

Capabilities

High-Level Parsing

The main parsing function that provides a simple interface for parsing Python code without needing to explicitly manage grammars.

def parse(code=None, **kwargs):
    """
    Parse Python code using a default grammar.
    
    Args:
        code (str | bytes, optional): Python source code to parse
        version (str, optional): Python version for grammar (e.g., "3.9")
        error_recovery (bool): Enable error recovery (default: True)
        path (str | Path, optional): File path for caching
        cache (bool): Enable caching (default: False)
        diff_cache (bool): Enable differential caching (default: False)
        cache_path (str | Path, optional): Custom cache directory
        file_io (FileIO, optional): File I/O handler
        start_symbol (str, optional): Grammar start symbol
        
    Returns:
        Module: Parsed syntax tree module
        
    Raises:
        ParserSyntaxError: If parsing fails and error_recovery is False
        TypeError: If neither code nor path provided
    """

Usage Examples

import parso

# Parse string code
module = parso.parse('x = 1 + 2')
expr_stmt = module.children[0]
print(expr_stmt.get_code())  # 'x = 1 + 2'

# Parse with specific Python version
module = parso.parse('match x:\n    case 1: pass', version="3.10")

# Parse from file path
module = parso.parse(path="/path/to/script.py")

# Parse with caching enabled
module = parso.parse('def foo(): pass', cache=True, path="example.py")

# Parse with error recovery disabled (raises on syntax errors)
try:
    module = parso.parse('invalid syntax here', error_recovery=False)
except parso.ParserSyntaxError as e:
    print(f"Syntax error: {e.message}")

Grammar Loading

Function to load Python grammars for specific versions, enabling more control over the parsing process and access to advanced features.

def load_grammar(*, version=None, path=None):
    """
    Load a Python grammar for a specific version.
    
    Args:
        version (str, optional): Python version string (e.g., "3.8", "3.10")
                               Defaults to current Python version
        path (str, optional): Path to custom grammar file
        
    Returns:
        PythonGrammar: Grammar instance for parsing
        
    Raises:
        NotImplementedError: If Python version is not supported
        FileNotFoundError: If custom grammar file not found
    """

Usage Examples

import parso

# Load grammar for current Python version
grammar = parso.load_grammar()

# Load grammar for specific version
grammar38 = parso.load_grammar(version="3.8")
grammar310 = parso.load_grammar(version="3.10")

# Parse with loaded grammar
module = grammar310.parse('match x:\n    case 1: print("one")')

# Check for syntax errors
errors = list(grammar.iter_errors(module))
if errors:
    for error in errors:
        print(f"Error at {error.start_pos}: {error.message}")

# Load custom grammar from file
custom_grammar = parso.load_grammar(path="/path/to/custom.txt")

Supported Python Versions

Parso supports Python grammar versions from 3.6 to 3.14:

  • 3.6: f-strings, variable annotations
  • 3.7: dataclasses syntax support
  • 3.8: positional-only parameters, walrus operator (:=)
  • 3.9: dictionary union operators, type hinting improvements
  • 3.10: match statements, union types (X | Y)
  • 3.11: exception groups, async comprehensions improvements
  • 3.12: f-string improvements, type parameter syntax
  • 3.13: type parameter syntax refinements
  • 3.14: latest supported version

Version-Specific Parsing

import parso

# Python 3.8 features - walrus operator
grammar38 = parso.load_grammar(version="3.8")
module = grammar38.parse('if (n := len(items)) > 5: pass')

# Python 3.10 features - match statements
grammar310 = parso.load_grammar(version="3.10")
module = grammar310.parse('''
match value:
    case 1:
        print("one")
    case 2 | 3:
        print("two or three")
    case _:
        print("other")
''')

# Use newer version for backwards compatibility
future_grammar = parso.load_grammar(version="4.0")  # Uses 3.14 grammar

Error Handling

Parse Errors

When error_recovery=False, parsing failures raise ParserSyntaxError:

import parso

try:
    # This will raise because of invalid syntax
    module = parso.parse('def (invalid): pass', error_recovery=False)
except parso.ParserSyntaxError as e:
    print(f"Syntax error: {e.message}")
    print(f"Error location: {e.error_leaf.start_pos}")
    print(f"Error token: {e.error_leaf.value}")

Error Recovery Mode

With error_recovery=True (default), parso can parse invalid code:

import parso

# Parse invalid code - still returns a tree
module = parso.parse('def broken(: pass')  # Missing parameter name

# Check for errors using grammar
grammar = parso.load_grammar()
module = grammar.parse('for x in: pass')  # Missing iterable
errors = list(grammar.iter_errors(module))

for error in errors:
    print(f"Error: {error.message} at {error.start_pos}")

Performance Considerations

Caching

Enable caching for better performance when parsing the same files repeatedly:

import parso

# Enable pickle caching (saves parsed trees to disk)
module = parso.parse(
    path="/path/to/large_file.py",
    cache=True,
    cache_path="/custom/cache/directory"  # Optional custom location
)

# Enable differential caching (only re-parse changed parts)
module = parso.parse(
    path="/path/to/file.py",
    diff_cache=True,
    cache=True  # Also enable regular caching
)

Memory Management

For processing many files, consider clearing the cache periodically:

import parso.cache

# Clear all caches
parso.cache.clear_cache()

# Clear only inactive cache files
parso.cache.clear_inactive_cache()

Integration Patterns

File Processing

import parso
from pathlib import Path

def process_python_files(directory):
    """Process all Python files in a directory."""
    for py_file in Path(directory).glob("**/*.py"):
        try:
            module = parso.parse(path=str(py_file), cache=True)
            # Process the parsed module
            yield py_file, module
        except Exception as e:
            print(f"Error parsing {py_file}: {e}")

# Usage
for file_path, module in process_python_files("/path/to/project"):
    print(f"Parsed {file_path}: {len(module.children)} top-level statements")

Code Analysis

import parso

def analyze_code(source_code, version="3.9"):
    """Analyze Python code for basic metrics."""
    grammar = parso.load_grammar(version=version)
    module = grammar.parse(source_code)
    
    # Count different node types
    functions = list(module.iter_funcdefs())
    classes = list(module.iter_classdefs())
    imports = list(module.iter_imports())
    
    # Check for syntax errors
    errors = list(grammar.iter_errors(module))
    
    return {
        'functions': len(functions),
        'classes': len(classes),
        'imports': len(imports),
        'errors': len(errors),
        'lines': len(module.get_code().splitlines())
    }

# Usage
stats = analyze_code('''
import os
import sys

class Example:
    def method(self):
        return 42

def function():
    pass
''')
print(stats)  # {'functions': 2, 'classes': 1, 'imports': 2, 'errors': 0, 'lines': 9}

Install with Tessl CLI

npx tessl i tessl/pypi-parso

docs

core-parsing.md

error-handling.md

grammar-system.md

index.md

python-elements.md

tokenization.md

tree-navigation.md

utilities.md

tile.json