0
# Utilities and Templates
1

2
File operations, path manipulation, template formatting, and various utility functions for safe file handling and flexible path generation. These utilities provide the foundation for beets' file management and path formatting capabilities.
3

4
## Capabilities
5

6
### File Operations
7

8
Safe file operations with cross-platform compatibility and error handling.
9

10
```python { .api }
11
def normpath(path: Union[str, bytes]) -> bytes:
12
    """
13
    Normalize a filesystem path to bytes representation.
14
    
15
    Parameters:
16
    - path: Filesystem path as string or bytes
17
    
18
    Returns:
19
    Normalized path as bytes
20
    """
21

22
def syspath(path: Union[str, bytes]) -> str:
23
    """
24
    Convert path to system's preferred string representation.
25
    
26
    Parameters:
27
    - path: Filesystem path as string or bytes
28
    
29
    Returns:
30
    Path as Unicode string suitable for system calls
31
    """
32

33
def bytestring_path(path: Union[str, bytes]) -> bytes:
34
    """
35
    Convert path to bytestring representation.
36
    
37
    Parameters:
38
    - path: Filesystem path as string or bytes
39
    
40
    Returns:
41
    Path as bytes using filesystem encoding
42
    """
43

44
def displayable_path(path: Union[str, bytes]) -> str:
45
    """
46
    Convert path to Unicode string safe for display.
47
    
48
    Parameters:
49
    - path: Filesystem path as string or bytes
50
    
51
    Returns:
52
    Unicode string representation of path
53
    """
54

55
def samefile(p1: Union[str, bytes], p2: Union[str, bytes]) -> bool:
56
    """
57
    Check if two paths refer to the same file.
58
    
59
    Parameters:
60
    - p1: First path to compare
61
    - p2: Second path to compare
62
    
63
    Returns:
64
    True if paths refer to the same file
65
    """
66
```
67

68
### Safe File Operations
69

70
Functions for moving, copying, and linking files with error handling.
71

72
```python { .api }
73
def move(src: Union[str, bytes], dest: Union[str, bytes]) -> None:
74
    """
75
    Safely move a file from source to destination.
76
    
77
    Parameters:
78
    - src: Source file path
79
    - dest: Destination file path
80
    
81
    Raises:
82
    FileOperationError: If move operation fails
83
    """
84

85
def copy(src: Union[str, bytes], dest: Union[str, bytes]) -> None:
86
    """
87
    Safely copy a file from source to destination.
88
    
89
    Parameters:
90
    - src: Source file path
91
    - dest: Destination file path
92
    
93
    Raises:
94
    FileOperationError: If copy operation fails
95
    """
96

97
def link(src: Union[str, bytes], dest: Union[str, bytes]) -> None:
98
    """
99
    Create a hard link from source to destination.
100
    
101
    Parameters:
102
    - src: Source file path
103
    - dest: Destination file path
104
    
105
    Raises:
106
    FileOperationError: If linking is not supported or fails
107
    """
108

109
def reflink(src: Union[str, bytes], dest: Union[str, bytes]) -> None:
110
    """
111
    Create a copy-on-write link (reflink) if supported.
112
    
113
    Parameters:
114
    - src: Source file path  
115
    - dest: Destination file path
116
    
117
    Raises:
118
    FileOperationError: If reflinking is not supported or fails
119
    """
120

121
def unique_path(path: Union[str, bytes]) -> Union[str, bytes]:
122
    """
123
    Generate a unique file path if the given path already exists.
124
    
125
    Parameters:
126
    - path: Desired file path
127
    
128
    Returns:
129
    Unique path (original or with numeric suffix)
130
    """
131
```
132

133
### Path Utilities
134

135
Functions for path manipulation and analysis.
136

137
```python { .api }
138
def ancestry(path: Union[str, bytes]) -> List[Union[str, bytes]]:
139
    """
140
    Get all parent directories of a path.
141
    
142
    Parameters:
143
    - path: Filesystem path
144
    
145
    Returns:
146
    List of parent directories from deepest to root
147
    """
148

149
def components(path: Union[str, bytes]) -> List[Union[str, bytes]]:
150
    """
151
    Split path into individual components.
152
    
153
    Parameters:
154
    - path: Filesystem path
155
    
156
    Returns:
157
    List of path components
158
    """
159

160
def as_string(value: Any) -> str:
161
    """
162
    Convert value to Unicode string safely.
163
    
164
    Parameters:
165
    - value: Value to convert (any type)
166
    
167
    Returns:
168
    Unicode string representation
169
    """
170
```
171

172
### Template System
173

174
Flexible template formatting system for generating paths and strings.
175

176
```python { .api }
177
class Template:
178
    """Template for string formatting with conditional logic and functions."""
179
    
180
    def __init__(self, template_string: str):
181
        """
182
        Initialize template from format string.
183
        
184
        Parameters:
185
        - template_string: Template format string with $field and %function{} syntax
186
        """
187
    
188
    def substitute(self, values: Dict[str, Any], functions: Dict[str, Callable] = None) -> str:
189
        """
190
        Substitute values into template.
191
        
192
        Parameters:
193
        - values: Dictionary mapping field names to values
194
        - functions: Optional dictionary of template functions
195
        
196
        Returns:
197
        Formatted string with substitutions applied
198
        """
199

200
def template(template_string: str) -> Template:
201
    """
202
    Create Template object from format string.
203
    
204
    Parameters:
205
    - template_string: Template format string
206
    
207
    Returns:
208
    Template object ready for substitution
209
    """
210
```
211

212
### Template Environment
213

214
Context for template evaluation with built-in functions.
215

216
```python { .api }
217
class Environment:
218
    """Template evaluation environment with built-in functions."""
219
    
220
    def __init__(self, values: Dict[str, Any]):
221
        """
222
        Initialize environment with field values.
223
        
224
        Parameters:
225
        - values: Dictionary mapping field names to values
226
        """
227
    
228
    def call_function(self, name: str, args: List[Any]) -> Any:
229
        """
230
        Call a template function by name.
231
        
232
        Parameters:
233
        - name: Function name
234
        - args: Function arguments
235
        
236
        Returns:
237
        Function result
238
        """
239
```
240

241
### Built-in Template Functions
242

243
Template functions available for path formatting and string manipulation.
244

245
```python { .api }
246
# String transformation functions
247
def tmpl_upper(text: str) -> str:
248
    """Convert text to uppercase."""
249

250
def tmpl_lower(text: str) -> str:
251
    """Convert text to lowercase."""
252

253
def tmpl_title(text: str) -> str:
254
    """Convert text to title case."""
255

256
# String manipulation functions  
257
def tmpl_left(text: str, length: int) -> str:
258
    """Take leftmost characters up to length."""
259

260
def tmpl_right(text: str, length: int) -> str:
261
    """Take rightmost characters up to length."""
262

263
def tmpl_if(condition: Any, then_value: Any, else_value: Any = '') -> Any:
264
    """Conditional expression (if condition then then_value else else_value)."""
265

266
def tmpl_ifdef(field: str, then_value: Any, else_value: Any = '') -> Any:
267
    """Conditional based on field existence (if field defined then then_value else else_value)."""
268

269
def tmpl_asciify(text: str) -> str:
270
    """Convert Unicode text to ASCII approximation."""
271

272
def tmpl_sanitize(text: str) -> str:
273
    """Remove/replace characters invalid in filenames."""
274
```
275

276
## Template Syntax and Usage
277

278
### Basic Field Substitution
279

280
```python
281
from beets.util.functemplate import template
282

283
# Basic field substitution
284
tmpl = template('$artist - $title')
285
result = tmpl.substitute({
286
    'artist': 'The Beatles',
287
    'title': 'Hey Jude'
288
})
289
# Result: "The Beatles - Hey Jude"
290

291
# Path template
292
path_tmpl = template('$albumartist/$album/$track $title')
293
path = path_tmpl.substitute({
294
    'albumartist': 'The Beatles',
295
    'album': 'Abbey Road', 
296
    'track': 5,
297
    'title': 'Here Comes the Sun'
298
})
299
# Result: "The Beatles/Abbey Road/05 Here Comes the Sun"
300
```
301

302
### Template Functions
303

304
```python
305
# String transformations
306
tmpl = template('%upper{$artist} - %lower{$title}')
307
result = tmpl.substitute({
308
    'artist': 'The Beatles',
309
    'title': 'HEY JUDE'
310
})
311
# Result: "THE BEATLES - hey jude"
312

313
# Conditional formatting
314
tmpl = template('$artist%if{$year, - $year}')
315
result = tmpl.substitute({
316
    'artist': 'The Beatles',
317
    'year': 1969
318
})
319
# Result: "The Beatles - 1969"
320

321
# String truncation
322
tmpl = template('%left{$title,20}')
323
result = tmpl.substitute({
324
    'title': 'Very Long Song Title That Needs Truncation'
325
})
326
# Result: "Very Long Song Title "
327
```
328

329
### Advanced Template Features
330

331
```python
332
# Nested conditionals
333
tmpl = template('%if{$albumartist,$albumartist,%if{$artist,$artist,Unknown Artist}}')
334

335
# Field existence checks
336
tmpl = template('%ifdef{$year,$year,0000}')
337

338
# Multiple functions
339
tmpl = template('%upper{%left{$artist,10}}')
340

341
# Filename sanitization
342
tmpl = template('%sanitize{$artist - $title}')
343
```
344

345
### Path Format Configuration
346

347
```python
348
from beets import config
349
from beets.util.functemplate import template
350

351
# Get path formats from configuration
352
path_formats = config['paths'].get(dict)
353

354
# Common path formats
355
formats = {
356
    'default': '$albumartist/$album/$track $title',
357
    'singleton': 'Non-Album/$artist - $title', 
358
    'comp': 'Compilations/$album/$track $title',
359
    'albumtype:soundtrack': 'Soundtracks/$album/$track $title'
360
}
361

362
# Apply formats
363
for pattern, format_str in formats.items():
364
    tmpl = template(format_str)
365
    # Use template for path generation
366
```
367

368
## File Operation Examples
369

370
### Safe File Moving
371

372
```python
373
from beets.util import move, copy, samefile
374
from beets.library import FileOperationError
375

376
def safe_file_operation(src_path, dest_path, operation='move'):
377
    """Safely perform file operations with error handling."""
378
    
379
    try:
380
        if samefile(src_path, dest_path):
381
            print("Source and destination are the same file")
382
            return
383
            
384
        if operation == 'move':
385
            move(src_path, dest_path)
386
            print(f"Moved: {src_path} -> {dest_path}")
387
        elif operation == 'copy':
388
            copy(src_path, dest_path)
389
            print(f"Copied: {src_path} -> {dest_path}")
390
            
391
    except FileOperationError as e:
392
        print(f"File operation failed: {e}")
393
        raise
394
```
395

396
### Path Generation
397

398
```python
399
from beets.util import unique_path, displayable_path
400
from beets.util.functemplate import template
401

402
def generate_item_path(item, library):
403
    """Generate destination path for a library item."""
404
    
405
    # Get path format template
406
    path_template = template(library.path_format_for_item(item))
407
    
408
    # Generate path
409
    dest_path = path_template.substitute(item.formatted())
410
    
411
    # Ensure path is unique
412
    dest_path = unique_path(dest_path)
413
    
414
    # Convert to displayable format
415
    display_path = displayable_path(dest_path)
416
    
417
    return dest_path, display_path
418
```
419

420
### Directory Creation
421

422
```python
423
import os
424
from beets.util import ancestry, displayable_path
425

426
def ensure_directory_exists(path):
427
    """Create directory and all parent directories if needed."""
428
    
429
    directory = os.path.dirname(path)
430
    
431
    if not os.path.exists(directory):
432
        try:
433
            os.makedirs(directory, exist_ok=True)
434
            print(f"Created directory: {displayable_path(directory)}")
435
        except OSError as e:
436
            print(f"Failed to create directory {displayable_path(directory)}: {e}")
437
            raise
438
```
439

440
## Art and Image Utilities
441

442
### Image Resizing
443

444
```python { .api }
445
class ArtResizer:
446
    """Utility for resizing and processing album artwork."""
447
    
448
    def __init__(self, quality: int = 95):
449
        """
450
        Initialize art resizer.
451
        
452
        Parameters:
453
        - quality: JPEG quality for output (1-100)
454
        """
455
    
456
    def resize(self, image_path: str, output_path: str, size: Tuple[int, int]) -> None:
457
        """
458
        Resize image to specified dimensions.
459
        
460
        Parameters:
461
        - image_path: Source image file path
462
        - output_path: Destination image file path  
463
        - size: Target (width, height) tuple
464
        """
465
    
466
    def thumbnail(self, image_path: str, output_path: str, size: int) -> None:
467
        """
468
        Create square thumbnail of image.
469
        
470
        Parameters:
471
        - image_path: Source image file path
472
        - output_path: Destination image file path
473
        - size: Thumbnail size (width and height)
474
        """
475
```
476

477
### Image Format Conversion
478

479
```python
480
from beets.util.artresizer import ArtResizer
481

482
def convert_artwork(src_path, dest_path, max_size=500):
483
    """Convert and resize artwork for embedding."""
484
    
485
    resizer = ArtResizer(quality=90)
486
    
487
    try:
488
        # Create thumbnail
489
        resizer.thumbnail(src_path, dest_path, max_size)
490
        print(f"Created artwork: {dest_path}")
491
        
492
    except Exception as e:
493
        print(f"Artwork conversion failed: {e}")
494
        raise
495
```
496

497
## Pipeline Processing
498

499
### Parallel Processing Utilities
500

501
```python { .api }
502
class Pipeline:
503
    """Parallel processing pipeline for import operations."""
504
    
505
    def __init__(self, stages: List[Callable]):
506
        """
507
        Initialize processing pipeline.
508
        
509
        Parameters:
510
        - stages: List of processing functions
511
        """
512
    
513
    def run_parallel(self, items: List[Any], threads: int = 4) -> List[Any]:
514
        """
515
        Process items through pipeline stages in parallel.
516
        
517
        Parameters:
518
        - items: List of items to process
519
        - threads: Number of worker threads
520
        
521
        Returns:
522
        List of processed items
523
        """
524
```
525

526
### Processing Example
527

528
```python
529
from beets.util.pipeline import Pipeline
530

531
def process_import_batch(items):
532
    """Process a batch of items through parallel pipeline."""
533
    
534
    def load_metadata(item):
535
        """Load metadata from file."""
536
        item.load()
537
        return item
538
    
539
    def tag_item(item):
540
        """Apply automatic tagging."""
541
        # Tagging logic here
542
        return item
543
    
544
    def write_tags(item):
545
        """Write tags back to file."""
546
        item.write()
547
        return item
548
    
549
    # Create processing pipeline
550
    pipeline = Pipeline([load_metadata, tag_item, write_tags])
551
    
552
    # Process items in parallel
553
    processed = pipeline.run_parallel(items, threads=4)
554
    
555
    return processed
556
```
557

558
## Error Handling
559

560
```python { .api }
561
class FileOperationError(Exception):
562
    """Raised when file operations fail."""
563
    
564
    def __init__(self, operation: str, path: str, error: Exception):
565
        """
566
        Initialize file operation error.
567
        
568
        Parameters:
569
        - operation: Operation that failed ('move', 'copy', etc.)
570
        - path: File path involved in operation
571
        - error: Underlying exception
572
        """
573

574
class HumanReadableError(Exception):
575
    """Base for errors with human-readable messages."""
576
    
577
    def log(self, logger) -> None:
578
        """Log error with appropriate level and formatting."""
579
```
580

581
### Error Handling Examples
582

583
```python
584
from beets.util import FileOperationError, displayable_path
585

586
def handle_file_errors(operation_func, *args):
587
    """Wrapper for file operations with error handling."""
588
    
589
    try:
590
        return operation_func(*args)
591
        
592
    except FileOperationError as e:
593
        print(f"File operation failed: {e}")
594
        print(f"Path: {displayable_path(e.path)}")
595
        raise
596
        
597
    except PermissionError as e:
598
        print(f"Permission denied: {displayable_path(e.filename)}")
599
        raise
600
        
601
    except OSError as e:
602
        print(f"System error: {e}")
603
        raise
604
```
605

606
This comprehensive utilities system provides the foundation for all file operations, path manipulation, and template formatting in beets, ensuring cross-platform compatibility and robust error handling.