0
# ImportMagic
1

2
A Python library for automated import management and symbol resolution. ImportMagic builds comprehensive symbol indexes from installed packages, analyzes Python source code to identify unresolved symbol references, and automatically generates appropriate import statements to resolve them.
3

4
## Package Information
5

6
- **Package Name**: importmagic
7
- **Language**: Python
8
- **Installation**: `pip install importmagic`
9

10
## Core Imports
11

12
```python
13
import importmagic
14
```
15

16
Specific functionality imports:
17

18
```python
19
from importmagic import SymbolIndex, Scope, Imports, get_update, update_imports
20
```
21

22
## Basic Usage
23

24
```python
25
import importmagic
26
import sys
27

28
# Build an index of available symbols
29
index = importmagic.SymbolIndex()
30
index.build_index(sys.path)
31

32
# Analyze source code for symbol usage
33
python_source = '''
34
import os
35
def example():
36
    basename = path.basename("/tmp/file.txt")  # unresolved: path
37
    return basename
38
'''
39

40
scope = importmagic.Scope.from_source(python_source)
41
unresolved, unreferenced = scope.find_unresolved_and_unreferenced_symbols()
42

43
# Get updated source with resolved imports
44
new_source = importmagic.update_imports(python_source, index, unresolved, unreferenced)
45
print(new_source)
46
# Output includes: from os import path
47
```
48

49
## Architecture
50

51
ImportMagic follows a three-phase approach to automated import management:
52

53
### 1. Symbol Indexing (`SymbolIndex`)
54
- **Purpose**: Build comprehensive catalogs of available symbols from Python packages
55
- **Process**: Recursively scans Python paths, parsing modules to extract exportable symbols
56
- **Scoring**: Assigns relevance scores based on module location (local > third-party > system > future)
57
- **Storage**: Hierarchical tree structure enabling fast symbol lookup and JSON serialization
58

59
### 2. Source Analysis (`Scope`)
60
- **Purpose**: Analyze Python source code to identify symbol usage patterns  
61
- **Process**: Uses AST (Abstract Syntax Tree) visitor pattern to traverse code structure
62
- **Tracking**: Maintains separate sets of defined symbols and referenced symbols across nested scopes
63
- **Resolution**: Identifies unresolved references (missing imports) and unreferenced imports (unused)
64

65
### 3. Import Management (`Imports`)
66
- **Purpose**: Generate and format import statements according to PEP8 conventions
67
- **Process**: Tokenizes existing imports, applies additions/removals, regenerates import block
68
- **Formatting**: Supports multiple import styles (parentheses, backslash) with configurable line lengths
69
- **Integration**: Seamlessly updates source code while preserving non-import content
70

71
### Component Relationships
72

73
```
74
SymbolIndex ──┐
75
              ├─→ Imports ──→ Updated Source Code
76
Source Code ──┘              ↗
77
   │                        │
78
   ↓                        │
79
 Scope ──→ Analysis Results ─┘
80
```
81

82
The workflow typically involves:
83
1. Building a `SymbolIndex` from available Python paths
84
2. Creating a `Scope` from source code to identify symbol usage
85
3. Using `Imports` to resolve unresolved symbols and clean up unreferenced imports
86
4. Generating updated source code with properly formatted import statements
87

88
This architecture enables both high-level convenience functions (`update_imports`) and fine-grained control over each phase for advanced use cases.
89

90
## Capabilities
91

92
### Symbol Indexing
93

94
Build searchable indexes of Python symbols from modules and packages for import resolution.
95

96
```python { .api }
97
class SymbolIndex:
98
    def __init__(self, name=None, parent=None, score=1.0, location='L', blacklist_re=None, locations=None):
99
        """
100
        Create a new symbol index.
101
        
102
        Parameters:
103
        - name: Optional name for this index node
104
        - parent: Parent SymbolIndex node
105
        - score: Score multiplier for symbols in this index
106
        - location: Location classification ('F', '3', 'S', 'L')
107
        - blacklist_re: Regex pattern for modules to exclude
108
        - locations: Library location mappings
109
        """
110
    
111
    def build_index(self, paths):
112
        """
113
        Build comprehensive symbol index from list of paths.
114
        
115
        Parameters:
116
        - paths: List of directory paths to index (typically sys.path)
117
        """
118
    
119
    def symbol_scores(self, symbol):
120
        """
121
        Find scored matches for a symbol name.
122
        
123
        Parameters:
124
        - symbol: Dotted symbol name to search for
125
        
126
        Returns:
127
        List of tuples (score, module, variable) ordered by score
128
        """
129
    
130
    def index_path(self, root):
131
        """
132
        Index a single path (file or directory).
133
        
134
        Parameters:
135
        - root: File path or package directory to index
136
        """
137
    
138
    def serialize(self, fd=None):
139
        """
140
        Serialize index to JSON format.
141
        
142
        Parameters:
143
        - fd: Optional file descriptor to write to
144
        
145
        Returns:
146
        JSON string if fd is None, otherwise writes to fd
147
        """
148
    
149
    @classmethod
150
    def deserialize(cls, file):
151
        """
152
        Load index from JSON file.
153
        
154
        Parameters:
155
        - file: File object to read from
156
        
157
        Returns:
158
        SymbolIndex instance
159
        """
160
    
161
    def index_source(self, filename, source):
162
        """
163
        Index symbols from Python source code.
164
        
165
        Parameters:
166
        - filename: Name or path of the source file
167
        - source: Python source code string
168
        """
169
    
170
    def index_file(self, module, filename):
171
        """
172
        Index symbols from a Python file.
173
        
174
        Parameters:
175
        - module: Module name for the file
176
        - filename: Path to the Python file
177
        """
178
    
179
    def index_builtin(self, name, location):
180
        """
181
        Index symbols from a built-in module.
182
        
183
        Parameters:
184
        - name: Built-in module name
185
        - location: Location classification for the module
186
        """
187
    
188
    def find(self, path):
189
        """
190
        Find index node for a dotted path.
191
        
192
        Parameters:
193
        - path: Dotted symbol path to find
194
        
195
        Returns:
196
        SymbolIndex node or None if not found
197
        """
198
    
199
    def location_for(self, path):
200
        """
201
        Get location classification for a symbol path.
202
        
203
        Parameters:
204
        - path: Dotted symbol path
205
        
206
        Returns:
207
        Location code string ('F', '3', 'S', 'L')
208
        """
209
    
210
    def add(self, name, score):
211
        """
212
        Add a symbol with score to current index node.
213
        
214
        Parameters:
215
        - name: Symbol name to add
216
        - score: Score value for the symbol
217
        """
218
    
219
    def add_explicit_export(self, name, score):
220
        """
221
        Add an explicitly exported symbol with score.
222
        
223
        Parameters:
224
        - name: Symbol name to export
225
        - score: Score value for the exported symbol
226
        """
227
    
228
    def enter(self, name, location='L', score=1.0):
229
        """
230
        Enter a sub-namespace context manager.
231
        
232
        Parameters:
233
        - name: Namespace name
234
        - location: Location classification (default 'L')
235
        - score: Score multiplier (default 1.0)
236
        
237
        Returns:
238
        Context manager yielding child SymbolIndex
239
        """
240
    
241
    def depth(self):
242
        """
243
        Get the depth of this index node in the tree.
244
        
245
        Returns:
246
        Integer depth from root (0 for root)
247
        """
248
    
249
    def path(self):
250
        """
251
        Get dotted path from root to this index node.
252
        
253
        Returns:
254
        Dotted string path
255
        """
256
    
257
    def boost(self):
258
        """
259
        Get boost multiplier for this location type.
260
        
261
        Returns:
262
        Float boost multiplier
263
        """
264
```
265

266
### Source Code Analysis
267

268
Analyze Python source code to identify symbol definitions, references, and scope relationships.
269

270
```python { .api }
271
class Scope:
272
    def __init__(self, parent=None, define_builtins=True, is_class=False):
273
        """
274
        Create a new scope for symbol tracking.
275
        
276
        Parameters:
277
        - parent: Parent scope (None for root scope)
278
        - define_builtins: Whether to include Python builtins
279
        - is_class: Whether this scope represents a class
280
        """
281
    
282
    @classmethod
283
    def from_source(cls, src, trace=False, define_builtins=True):
284
        """
285
        Create scope by analyzing Python source code.
286
        
287
        Parameters:
288
        - src: Python source code string or AST node
289
        - trace: Enable tracing for debugging
290
        - define_builtins: Whether to include Python builtins
291
        
292
        Returns:
293
        Scope instance with analyzed symbol information
294
        """
295
    
296
    def find_unresolved_and_unreferenced_symbols(self):
297
        """
298
        Find symbols that need import resolution or removal.
299
        
300
        Returns:
301
        Tuple of (unresolved_set, unreferenced_set)
302
        """
303
    
304
    def define(self, name):
305
        """
306
        Mark a symbol as defined in current scope.
307
        
308
        Parameters:
309
        - name: Symbol name to define
310
        """
311
    
312
    def reference(self, name):
313
        """
314
        Mark a symbol as referenced.
315
        
316
        Parameters:
317
        - name: Symbol name that was referenced
318
        """
319
    
320
    def enter(self, is_class=False):
321
        """
322
        Enter a child scope context manager.
323
        
324
        Parameters:
325
        - is_class: Whether the child scope represents a class
326
        
327
        Returns:
328
        Context manager yielding child Scope
329
        """
330
    
331
    def start_symbol(self):
332
        """
333
        Start building a compound symbol context manager.
334
        
335
        Returns:
336
        Context manager for compound symbol construction
337
        """
338
    
339
    def start_definition(self):
340
        """
341
        Start a symbol definition context manager.
342
        
343
        Returns:
344
        Context manager for symbol definition
345
        """
346
    
347
    def start_reference(self):
348
        """
349
        Start a symbol reference context manager.
350
        
351
        Returns:
352
        Context manager for symbol reference
353
        """
354
    
355
    def extend_symbol(self, segment, extend_only=False):
356
        """
357
        Extend current compound symbol with a segment.
358
        
359
        Parameters:
360
        - segment: Symbol segment to add
361
        - extend_only: If True, only extend without creating reference
362
        """
363
    
364
    def end_symbol(self):
365
        """
366
        End current compound symbol and mark as referenced.
367
        """
368
    
369
    def flush_symbol(self):
370
        """
371
        Flush any pending symbol operations.
372
        """
373
```
374

375
### Import Management
376

377
Manage import statements with parsing, modification, and PEP8-compliant formatting.
378

379
```python { .api }
380
class Imports:
381
    def __init__(self, index, source):
382
        """
383
        Create import manager for source code.
384
        
385
        Parameters:
386
        - index: SymbolIndex for symbol resolution
387
        - source: Python source code string
388
        """
389
    
390
    def add_import(self, name, alias=None):
391
        """
392
        Add a direct module import.
393
        
394
        Parameters:
395
        - name: Module name to import
396
        - alias: Optional import alias
397
        """
398
    
399
    def add_import_from(self, module, name, alias=None):
400
        """
401
        Add a from-module import.
402
        
403
        Parameters:
404
        - module: Module to import from
405
        - name: Symbol name to import
406
        - alias: Optional import alias
407
        """
408
    
409
    def remove(self, references):
410
        """
411
        Remove imports by referenced names.
412
        
413
        Parameters:
414
        - references: Set of symbol names to remove
415
        """
416
    
417
    def get_update(self):
418
        """
419
        Get import block update information.
420
        
421
        Returns:
422
        Tuple of (start_line, end_line, import_text)
423
        """
424
    
425
    def update_source(self):
426
        """
427
        Return complete updated source code.
428
        
429
        Returns:
430
        Updated Python source code string
431
        """
432
    
433
    @classmethod
434
    def set_style(cls, **kwargs):
435
        """
436
        Configure import formatting style.
437
        
438
        Parameters:
439
        - multiline: Multiline style ('parentheses' or 'backslash')
440
        - max_columns: Maximum line length for imports
441
        """
442

443
class Import:
444
    def __init__(self, location, name, alias):
445
        """
446
        Represent a single import statement.
447
        
448
        Parameters:
449
        - location: Import location classification
450
        - name: Module or symbol name
451
        - alias: Import alias or None
452
        """
453
    
454
    def __hash__(self):
455
        """
456
        Hash function for Import objects.
457
        
458
        Returns:
459
        Hash value based on location, name, and alias
460
        """
461
    
462
    def __eq__(self, other):
463
        """
464
        Equality comparison for Import objects.
465
        
466
        Parameters:
467
        - other: Other Import object to compare
468
        
469
        Returns:
470
        True if imports are equal
471
        """
472
    
473
    def __lt__(self, other):
474
        """
475
        Less-than comparison for sorting Import objects.
476
        
477
        Parameters:
478
        - other: Other Import object to compare
479
        
480
        Returns:
481
        True if this import should sort before other
482
        """
483
```
484

485
### High-Level Import Functions
486

487
Convenient functions for automated import management without manual class instantiation.
488

489
```python { .api }
490
def get_update(src, index, unresolved, unreferenced):
491
    """
492
    Generate import block update for source code.
493
    
494
    Parameters:
495
    - src: Python source code string
496
    - index: SymbolIndex instance for symbol resolution
497
    - unresolved: Set of unresolved symbol names
498
    - unreferenced: Set of unreferenced import names
499
    
500
    Returns:
501
    Tuple of (start_line, end_line, import_block_text)
502
    """
503

504
def update_imports(src, index, unresolved, unreferenced):
505
    """
506
    Update source code with resolved imports.
507
    
508
    Parameters:
509
    - src: Python source code string
510
    - index: SymbolIndex instance for symbol resolution
511
    - unresolved: Set of unresolved symbol names
512
    - unreferenced: Set of unreferenced import names
513
    
514
    Returns:
515
    Updated Python source code string
516
    """
517
```
518

519
## Types
520

521
```python { .api }
522
# Location Classifications
523
LOCATION_ORDER = 'FS3L'  # Future, System, Third-party, Local
524

525
# Symbol Index Locations
526
LOCATIONS = {
527
    'F': 'Future',      # __future__ imports
528
    '3': 'Third party', # Third-party packages  
529
    'S': 'System',      # Standard library
530
    'L': 'Local'        # Local modules
531
}
532

533
# Location Score Boosts
534
LOCATION_BOOSTS = {
535
    '3': 1.2,  # Third-party packages
536
    'L': 1.5,  # Local modules
537
}
538

539
# Package Aliases with Score Boosts
540
PACKAGE_ALIASES = {
541
    'os.path': ('posixpath', 1.2),  # Prefer os.path over posixpath/ntpath
542
    'os': ('os', 1.2),             # Boost os due to heavy aliasing
543
}
544

545
# Built-in Modules (treated specially during indexing)
546
BUILTIN_MODULES = sys.builtin_module_names + ('os',)
547

548
# Default module blacklist pattern
549
DEFAULT_BLACKLIST_RE = re.compile(r'\btest[s]?|test[s]?\b', re.I)
550

551
# Builtin Symbol Sets
552
GLOBALS = ['__name__', '__file__', '__loader__', '__package__', '__path__']
553
PYTHON3_BUILTINS = ['PermissionError']
554
ALL_BUILTINS = set(dir(__builtin__)) | set(GLOBALS) | set(PYTHON3_BUILTINS)
555

556
# Iterator class for token processing
557
class Iterator:
558
    def __init__(self, tokens, start=None, end=None):
559
        """
560
        Iterator for processing token sequences.
561
        
562
        Parameters:
563
        - tokens: List of tokens to iterate over
564
        - start: Starting index (default 0)
565
        - end: Ending index (default len(tokens))
566
        """
567
    
568
    def next(self):
569
        """
570
        Get next token and advance cursor.
571
        
572
        Returns:
573
        Tuple of (index, token) or (None, None) if exhausted
574
        """
575
    
576
    def peek(self):
577
        """
578
        Get current token without advancing cursor.
579
        
580
        Returns:
581
        Current token or None if exhausted
582
        """
583
    
584
    def until(self, type):
585
        """
586
        Collect tokens until specified type is found.
587
        
588
        Parameters:
589
        - type: Token type to search for
590
        
591
        Returns:
592
        List of (index, token) tuples
593
        """
594
    
595
    def rewind(self):
596
        """
597
        Move cursor back one position.
598
        """
599
```
600

601
## Advanced Usage Examples
602

603
### Fine-Grained Import Control
604

605
```python
606
import importmagic
607
import sys
608

609
# Build index
610
index = importmagic.SymbolIndex()
611
index.build_index(sys.path)
612

613
# Analyze source
614
python_source = '''
615
def example():
616
    result = basename("/tmp/file.txt")
617
    data = json.loads('{"key": "value"}')
618
    return result, data
619
'''
620

621
scope = importmagic.Scope.from_source(python_source)
622
unresolved, unreferenced = scope.find_unresolved_and_unreferenced_symbols()
623

624
# Manual import management
625
imports = importmagic.Imports(index, python_source)
626
imports.remove(unreferenced)
627

628
# Add specific imports with control over selection
629
for symbol in unresolved:
630
    scores = index.symbol_scores(symbol)
631
    if scores:
632
        score, module, variable = scores[0]  # Take highest scored match
633
        if variable is None:
634
            imports.add_import(module)  # Direct module import
635
        else:
636
            imports.add_import_from(module, variable)  # From-import
637

638
updated_source = imports.update_source()
639
```
640

641
### Custom Import Formatting
642

643
```python
644
# Configure import style
645
importmagic.Imports.set_style(
646
    multiline='parentheses',  # Use parentheses for multiline imports
647
    max_columns=80           # Maximum line length
648
)
649

650
# Process source with custom formatting
651
imports = importmagic.Imports(index, source)
652
# ... add/remove imports
653
formatted_source = imports.update_source()
654
```
655

656
### Index Persistence
657

658
```python
659
# Save index to file
660
index = importmagic.SymbolIndex()
661
index.build_index(sys.path)
662
with open('symbol_index.json', 'w') as f:
663
    index.serialize(f)
664

665
# Load index from file
666
with open('symbol_index.json', 'r') as f:
667
    index = importmagic.SymbolIndex.deserialize(f)
668
```
669

670
## Error Handling
671

672
ImportMagic handles various error conditions gracefully:
673

674
- **Malformed source code**: AST parsing errors are caught and logged
675
- **Missing modules**: Unavailable imports are skipped during indexing
676
- **Circular imports**: Detected and avoided during symbol resolution
677
- **Invalid symbols**: Malformed symbol names are filtered out
678

679
The library is designed to be robust for use in development tools and IDEs where source code may be incomplete or contain syntax errors.