0
# Utilities
1
2
Helper functions for image processing, file operations, and common tasks used throughout Sphinx-Gallery. These utilities provide core functionality for image manipulation, file handling, and system operations.
3
4
## Capabilities
5
6
### Image Processing
7
8
Function for scaling images while maintaining aspect ratio.
9
10
```python { .api }
11
def scale_image(in_fname, out_fname, max_width, max_height):
12
"""
13
Scales images while maintaining aspect ratio.
14
15
Resizes an image to fit within the specified maximum dimensions
16
while preserving the original aspect ratio. Uses high-quality
17
resampling for optimal results.
18
19
Parameters:
20
- in_fname: str, input image filename
21
- out_fname: str, output image filename
22
- max_width: int, maximum width in pixels
23
- max_height: int, maximum height in pixels
24
25
Returns:
26
None
27
"""
28
```
29
30
#### Usage Example
31
32
```python
33
from sphinx_gallery.utils import scale_image
34
35
# Scale image to fit within 200x200 pixels
36
scale_image(
37
'original_plot.png',
38
'thumbnail_plot.png',
39
200,
40
200
41
)
42
43
# Create multiple sizes
44
scale_image('plot.png', 'plot_small.png', 150, 150)
45
scale_image('plot.png', 'plot_medium.png', 300, 300)
46
scale_image('plot.png', 'plot_large.png', 600, 600)
47
```
48
49
### File Hash Operations
50
51
Functions for computing file hashes and checksums.
52
53
```python { .api }
54
def md5sum_file(filename):
55
"""
56
Calculate MD5 hash of a file.
57
58
Computes the MD5 checksum of a file for content verification
59
and change detection.
60
61
Parameters:
62
- filename: str, path to file
63
64
Returns:
65
str: MD5 hash as hexadecimal string
66
"""
67
68
def get_md5sum(src_file, mode='md5'):
69
"""
70
Get file hash using specified algorithm.
71
72
Parameters:
73
- src_file: str, path to source file
74
- mode: str, hash algorithm ('md5', 'sha1', 'sha256')
75
76
Returns:
77
str: File hash as hexadecimal string
78
"""
79
```
80
81
#### Usage Example
82
83
```python
84
from sphinx_gallery.utils import md5sum_file, get_md5sum
85
86
# Check if file has changed
87
old_hash = md5sum_file('example.py')
88
# ... file potentially modified ...
89
new_hash = md5sum_file('example.py')
90
91
if old_hash != new_hash:
92
print("File has been modified")
93
94
# Use different hash algorithms
95
md5_hash = get_md5sum('data.txt', 'md5')
96
sha256_hash = get_md5sum('data.txt', 'sha256')
97
```
98
99
### File Path Operations
100
101
Functions for manipulating file paths and extensions.
102
103
```python { .api }
104
def replace_py_ipynb(fname):
105
"""
106
Replace .py extension with .ipynb.
107
108
Converts Python script filename to Jupyter notebook filename.
109
110
Parameters:
111
- fname: str, filename with .py extension
112
113
Returns:
114
str: Filename with .ipynb extension
115
"""
116
```
117
118
#### Usage Example
119
120
```python
121
from sphinx_gallery.utils import replace_py_ipynb
122
123
# Convert script name to notebook name
124
script_name = 'plot_example.py'
125
notebook_name = replace_py_ipynb(script_name)
126
# Result: 'plot_example.ipynb'
127
128
# Handle paths
129
script_path = '/examples/advanced/plot_demo.py'
130
notebook_path = replace_py_ipynb(script_path)
131
# Result: '/examples/advanced/plot_demo.ipynb'
132
```
133
134
### String and Text Processing
135
136
Functions for processing text and string content.
137
138
```python { .api }
139
def _complete_chunk(chunk_lines, lang):
140
"""
141
Complete and validate code chunks.
142
143
Parameters:
144
- chunk_lines: list, lines of code
145
- lang: str, programming language
146
147
Returns:
148
str: Completed code chunk
149
"""
150
151
def _get_docstring_and_rest(filename):
152
"""
153
Extract docstring and remaining code from Python file.
154
155
Parameters:
156
- filename: str, path to Python file
157
158
Returns:
159
tuple: (docstring, remaining_code)
160
"""
161
```
162
163
### Memory and Performance Utilities
164
165
Functions for monitoring memory usage and performance.
166
167
```python { .api }
168
def _get_memory_usage():
169
"""
170
Get current memory usage.
171
172
Returns:
173
float: Memory usage in MB
174
"""
175
176
def optipng(fname, args=None):
177
"""
178
Optimize PNG image using optipng.
179
180
Parameters:
181
- fname: str, PNG filename to optimize
182
- args: list, additional optipng arguments
183
184
Returns:
185
None
186
"""
187
```
188
189
### System Integration
190
191
Functions for interacting with the system and external tools.
192
193
```python { .api }
194
def _get_image_ext(image_path):
195
"""
196
Get image file extension.
197
198
Parameters:
199
- image_path: str, path to image file
200
201
Returns:
202
str: File extension (e.g., '.png', '.jpg')
203
"""
204
205
def _has_optipng():
206
"""
207
Check if optipng is available on the system.
208
209
Returns:
210
bool: True if optipng is available
211
"""
212
```
213
214
## Advanced Utilities
215
216
### Custom Image Processing
217
218
```python
219
from sphinx_gallery.utils import scale_image
220
from PIL import Image
221
import os
222
223
def create_responsive_images(src_image, base_name, sizes):
224
"""
225
Create multiple sizes of an image for responsive display.
226
227
Parameters:
228
- src_image: str, source image path
229
- base_name: str, base name for output files
230
- sizes: list, list of (width, height) tuples
231
"""
232
for i, (width, height) in enumerate(sizes):
233
suffix = f"_{width}x{height}" if i > 0 else ""
234
output_name = f"{base_name}{suffix}.png"
235
scale_image(src_image, output_name, width, height)
236
237
# Usage
238
create_responsive_images(
239
'plot_large.png',
240
'plot',
241
[(200, 200), (400, 400), (800, 800)]
242
)
243
```
244
245
### Batch File Processing
246
247
```python
248
from sphinx_gallery.utils import md5sum_file
249
import os
250
import json
251
252
def create_file_manifest(directory):
253
"""
254
Create manifest of all files with their hashes.
255
256
Parameters:
257
- directory: str, directory to process
258
259
Returns:
260
dict: Mapping of filenames to MD5 hashes
261
"""
262
manifest = {}
263
264
for root, dirs, files in os.walk(directory):
265
for file in files:
266
if file.endswith(('.py', '.png', '.jpg')):
267
filepath = os.path.join(root, file)
268
relative_path = os.path.relpath(filepath, directory)
269
manifest[relative_path] = md5sum_file(filepath)
270
271
return manifest
272
273
# Usage
274
manifest = create_file_manifest('examples/')
275
with open('file_manifest.json', 'w') as f:
276
json.dump(manifest, f, indent=2)
277
```
278
279
### Image Optimization Pipeline
280
281
```python
282
from sphinx_gallery.utils import scale_image, optipng, _has_optipng
283
import os
284
285
def optimize_gallery_images(image_dir, thumbnail_size=(200, 200)):
286
"""
287
Optimize all images in gallery directory.
288
289
Parameters:
290
- image_dir: str, directory containing images
291
- thumbnail_size: tuple, maximum thumbnail dimensions
292
"""
293
294
for filename in os.listdir(image_dir):
295
if filename.endswith('.png'):
296
filepath = os.path.join(image_dir, filename)
297
298
# Create thumbnail
299
thumb_name = filename.replace('.png', '_thumb.png')
300
thumb_path = os.path.join(image_dir, thumb_name)
301
scale_image(filepath, thumb_path, *thumbnail_size)
302
303
# Optimize with optipng if available
304
if _has_optipng():
305
optipng(filepath)
306
optipng(thumb_path)
307
308
print(f"Processed {filename}")
309
310
# Usage
311
optimize_gallery_images('_build/html/_images/')
312
```
313
314
### File Change Detection
315
316
```python
317
from sphinx_gallery.utils import md5sum_file
318
import pickle
319
import os
320
321
class FileChangeTracker:
322
"""Track file changes using MD5 hashes."""
323
324
def __init__(self, cache_file='.file_hashes.pkl'):
325
self.cache_file = cache_file
326
self.hashes = self._load_hashes()
327
328
def _load_hashes(self):
329
"""Load cached hashes from file."""
330
if os.path.exists(self.cache_file):
331
with open(self.cache_file, 'rb') as f:
332
return pickle.load(f)
333
return {}
334
335
def _save_hashes(self):
336
"""Save hashes to cache file."""
337
with open(self.cache_file, 'wb') as f:
338
pickle.dump(self.hashes, f)
339
340
def has_changed(self, filepath):
341
"""Check if file has changed since last check."""
342
current_hash = md5sum_file(filepath)
343
old_hash = self.hashes.get(filepath)
344
345
changed = old_hash != current_hash
346
self.hashes[filepath] = current_hash
347
348
return changed
349
350
def save(self):
351
"""Save current state to disk."""
352
self._save_hashes()
353
354
# Usage
355
tracker = FileChangeTracker()
356
357
for example_file in os.listdir('examples/'):
358
if example_file.endswith('.py'):
359
filepath = os.path.join('examples/', example_file)
360
if tracker.has_changed(filepath):
361
print(f"File changed: {example_file}")
362
363
tracker.save()
364
```
365
366
### Error Handling Utilities
367
368
```python
369
import functools
370
import logging
371
372
def safe_image_operation(operation_name):
373
"""
374
Decorator for safe image operations with error handling.
375
376
Parameters:
377
- operation_name: str, name of the operation for logging
378
"""
379
def decorator(func):
380
@functools.wraps(func)
381
def wrapper(*args, **kwargs):
382
try:
383
return func(*args, **kwargs)
384
except Exception as e:
385
logging.warning(f"{operation_name} failed: {e}")
386
return None
387
return wrapper
388
return decorator
389
390
@safe_image_operation("Image scaling")
391
def safe_scale_image(in_fname, out_fname, max_width, max_height):
392
"""Safe wrapper around scale_image."""
393
from sphinx_gallery.utils import scale_image
394
return scale_image(in_fname, out_fname, max_width, max_height)
395
396
# Usage
397
result = safe_scale_image('input.png', 'output.png', 200, 200)
398
if result is None:
399
print("Image scaling failed")
400
```
401
402
## Integration Patterns
403
404
### With Sphinx Build Process
405
406
```python
407
def process_images_during_build(app, env, updated_docs, updated_assets):
408
"""Process images during Sphinx build."""
409
410
from sphinx_gallery.utils import scale_image
411
412
image_dir = os.path.join(app.outdir, '_images')
413
414
if os.path.exists(image_dir):
415
for image_file in os.listdir(image_dir):
416
if image_file.endswith('.png'):
417
# Create thumbnail version
418
base_name = image_file.replace('.png', '')
419
thumb_file = f"{base_name}_thumb.png"
420
421
scale_image(
422
os.path.join(image_dir, image_file),
423
os.path.join(image_dir, thumb_file),
424
150, 150
425
)
426
427
# In conf.py
428
def setup(app):
429
app.connect('env-updated', process_images_during_build)
430
```
431
432
### Performance Monitoring
433
434
```python
435
from sphinx_gallery.utils import _get_memory_usage
436
import time
437
import functools
438
439
def monitor_performance(func):
440
"""Decorator to monitor function performance."""
441
@functools.wraps(func)
442
def wrapper(*args, **kwargs):
443
start_time = time.time()
444
start_memory = _get_memory_usage()
445
446
result = func(*args, **kwargs)
447
448
end_time = time.time()
449
end_memory = _get_memory_usage()
450
451
print(f"{func.__name__}:")
452
print(f" Time: {end_time - start_time:.2f}s")
453
print(f" Memory: {end_memory - start_memory:.1f}MB")
454
455
return result
456
return wrapper
457
458
@monitor_performance
459
def process_large_gallery():
460
"""Example of monitored function."""
461
# Gallery processing code here
462
pass
463
```
464
465
## Best Practices
466
467
### Image Processing
468
- Use appropriate image formats (PNG for diagrams, JPEG for photos)
469
- Optimize images for web display while maintaining quality
470
- Create multiple sizes for responsive design
471
- Cache processed images to avoid reprocessing
472
473
### File Operations
474
- Use hash-based change detection for efficiency
475
- Implement proper error handling for file operations
476
- Clean up temporary files after processing
477
- Use atomic operations for critical file updates
478
479
### Performance
480
- Monitor memory usage for large galleries
481
- Use lazy loading for expensive operations
482
- Cache computed results when possible
483
- Profile bottlenecks in custom utilities