0
# Utilities and Specialized Extensions
1

2
Utility extensions, content processing tools, and specialized functionality including snippet inclusion, HTML processing, escape handling, tracked changes, enhanced markdown extra, and development utilities.
3

4
## Capabilities
5

6
### Snippets (File Inclusion)
7

8
Include external file content and code snippets with support for URL fetching, content filtering, and nested inclusions.
9

10
```python { .api }
11
def makeExtension(**kwargs):
12
    """
13
    Create Snippets extension for external content inclusion.
14
    
15
    Configuration:
16
    - base_path: str - Base path for relative file resolution
17
    - encoding: str - File encoding ('utf-8')
18
    - check_paths: bool - Validate file paths (False)
19
    - auto_append: list - Automatically append file extensions
20
    - url_timeout: float - URL fetch timeout in seconds (10.0)
21
    - url_max_size: int - Maximum URL content size (1MB)
22
    - url_request_headers: dict - Custom HTTP headers
23
    
24
    Returns:
25
    SnippetsExtension instance
26
    """
27

28
class SnippetsExtension(Extension):
29
    """External content inclusion extension."""
30

31
class SnippetsPreprocessor(Preprocessor):
32
    """Snippet inclusion preprocessor."""
33
```
34

35
**Usage Example:**
36

37
```python
38
import markdown
39

40
md = markdown.Markdown(extensions=[
41
    'pymdownx.snippets'
42
], extension_configs={
43
    'pymdownx.snippets': {
44
        'base_path': './docs',
45
        'check_paths': True,
46
        'auto_append': ['.md', '.txt']
47
    }
48
})
49

50
text = '''
51
# Documentation
52

53
--8<-- "common/header.md"
54

55
## Code Example
56

57
--8<-- "examples/hello.py"
58

59
## From URL
60

61
--8<-- "https://example.com/snippet.txt"
62
'''
63
html = md.convert(text)
64
```
65

66
### StripHTML (HTML Processing)
67

68
Remove or escape HTML tags from markdown content with configurable tag handling and content preservation.
69

70
```python { .api }
71
def makeExtension(**kwargs):
72
    """
73
    Create StripHTML extension for HTML tag processing.
74
    
75
    Configuration:
76
    - strip_comments: bool - Remove HTML comments (True)
77
    - strip_js_on_attributes: bool - Remove JavaScript event attributes (True)
78
    - strip_attributes: list - Attributes to remove from all tags
79
    - strip_js_on_attributes: bool - Remove JavaScript event handlers (True)
80
    
81
    Returns:
82
    StripHtmlExtension instance
83
    """
84

85
class StripHtmlExtension(Extension):
86
    """HTML stripping and processing extension."""
87

88
class StripHtmlPostprocessor(Postprocessor):
89
    """HTML processing postprocessor."""
90
```
91

92
**Usage Example:**
93

94
```python
95
import markdown
96

97
md = markdown.Markdown(extensions=[
98
    'pymdownx.striphtml'
99
], extension_configs={
100
    'pymdownx.striphtml': {
101
        'strip_comments': True,
102
        'strip_js_on_attributes': True,
103
        'strip_attributes': ['onclick', 'onload']
104
    }
105
})
106

107
text = '''
108
<div onclick="alert('xss')">Safe content</div>
109
<!-- This comment will be removed -->
110
<p>Regular paragraph</p>
111
'''
112
html = md.convert(text)  # JavaScript and comments removed
113
```
114

115
### EscapeAll (Universal Escaping)
116

117
Escape any character with backslash, providing universal escape functionality beyond standard markdown escaping.
118

119
```python { .api }
120
def makeExtension(**kwargs):
121
    """
122
    Create EscapeAll extension for universal character escaping.
123
    
124
    Configuration:
125
    - hardbreak: bool - Treat escaped newlines as hard breaks (False)
126
    - nbsp: bool - Treat escaped spaces as non-breaking spaces (False)
127
    
128
    Returns:
129
    EscapeAllExtension instance
130
    """
131

132
class EscapeAllExtension(Extension):
133
    """Universal character escaping extension."""
134

135
class EscapeAllPattern(InlineProcessor):
136
    """Escape pattern processor."""
137
```
138

139
**Usage Example:**
140

141
```python
142
import markdown
143

144
md = markdown.Markdown(extensions=['pymdownx.escapeall'])
145

146
text = r'''
147
Escaped \* asterisk doesn't create emphasis.
148
Escaped \# hash doesn't create header.
149
Escaped \[ bracket doesn't start link.
150
Escaped \\ backslash shows literal backslash.
151
'''
152
html = md.convert(text)
153
```
154

155
### Critic (CriticMarkup)
156

157
CriticMarkup support for tracked changes, comments, and collaborative editing with acceptance/rejection modes.
158

159
```python { .api }
160
def makeExtension(**kwargs):
161
    """
162
    Create Critic extension for CriticMarkup support.
163
    
164
    Configuration:
165
    - mode: str - Processing mode ('view', 'accept', 'reject')
166
    
167
    Returns:
168
    CriticExtension instance
169
    """
170

171
class CriticExtension(Extension):
172
    """CriticMarkup tracked changes extension."""
173

174
class CriticViewPreprocessor(Preprocessor):
175
    """Critic marks preprocessor for view mode."""
176

177
class CriticsPostprocessor(Postprocessor):
178
    """Critic output postprocessor."""
179

180
class CriticStash:
181
    """Critic marks stash manager."""
182
```
183

184
**Usage Example:**
185

186
```python
187
import markdown
188

189
# View mode (default) - shows all marks
190
md = markdown.Markdown(extensions=['pymdownx.critic'])
191

192
# Accept mode - applies additions, removes deletions
193
md_accept = markdown.Markdown(extensions=[
194
    'pymdownx.critic'
195
], extension_configs={
196
    'pymdownx.critic': {'mode': 'accept'}
197
})
198

199
# Reject mode - removes additions, keeps deletions  
200
md_reject = markdown.Markdown(extensions=[
201
    'pymdownx.critic'
202
], extension_configs={
203
    'pymdownx.critic': {'mode': 'reject'}
204
})
205

206
text = '''
207
{++Addition++} and {--deletion--} marks.
208

209
{~~Substitution~>replacement~~} example.
210

211
{>>Comment<<} annotation.
212

213
{==Highlighting==}{>>comment<<} with comment.
214
'''
215
html = md.convert(text)
216
```
217

218
### Extra (Enhanced Markdown Extra)
219

220
Enhanced version of Python-Markdown Extra with additional features and improved functionality.
221

222
```python { .api }
223
def makeExtension(**kwargs):
224
    """
225
    Create Extra extension for enhanced Markdown Extra.
226
    
227
    Returns:
228
    ExtraExtension instance
229
    """
230

231
class ExtraExtension(Extension):
232
    """Enhanced Markdown Extra extension."""
233

234
# Included extensions list
235
extra_extensions = [
236
    'markdown.extensions.extra',
237
    'pymdownx.betterem',
238
    'pymdownx.superfences',
239
    # ... additional extensions
240
]
241

242
# Extension configurations
243
extra_extension_configs = {
244
    'markdown.extensions.extra': {},
245
    'pymdownx.betterem': {},
246
    'pymdownx.superfences': {},
247
    # ... configuration options
248
}
249
```
250

251
**Usage Example:**
252

253
```python
254
import markdown
255

256
# Use PyMdown Extra instead of standard Extra
257
md = markdown.Markdown(extensions=['pymdownx.extra'])
258

259
text = '''
260
# Enhanced Extra Features
261

262
- [x] Task lists
263
- [ ] Better emphasis
264
- Code fences with highlighting
265

266
```python
267
def hello():
268
    print("Enhanced code blocks!")
269
```
270

271
*Better* **emphasis** handling.
272
'''
273
html = md.convert(text)
274
```
275
276
### Slugs (Header Slugification)
277

278
Advanced header ID generation and slugification with Unicode support and customization options.
279

280
```python { .api }
281
def makeExtension(**kwargs):
282
    """
283
    Create Slugs extension for header ID generation.
284
    
285
    Configuration:
286
    - slugify: callable - Custom slugify function
287
    - case: str - Case handling ('lower', 'upper', 'none')
288
    
289
    Returns:
290
    SlugsExtension instance
291
    """
292

293
class SlugsExtension(Extension):
294
    """Header slugification extension."""
295
```
296

297
**Usage Example:**
298

299
```python
300
import markdown
301

302
# Custom slugify function
303
def custom_slugify(text, separator):
304
    return text.lower().replace(' ', separator)
305

306
md = markdown.Markdown(extensions=[
307
    'pymdownx.slugs'
308
], extension_configs={
309
    'pymdownx.slugs': {
310
        'slugify': custom_slugify,
311
        'case': 'lower'
312
    }
313
})
314

315
text = '''
316
# Main Header
317
## Sub Header with Spaces
318
### Unicode Header 中文
319
'''
320
html = md.convert(text)
321
```
322

323
## Utility Functions and Classes
324

325
### Common Utilities
326

327
Shared utilities for extension development and markdown processing.
328

329
```python { .api }
330
# From pymdownx.util
331
class PatternSequenceProcessor(InlineProcessor):
332
    """
333
    Base class for complex pattern processing.
334
    
335
    Handles sequential pattern matching with configurable behavior.
336
    """
337

338
class PatSeqItem:
339
    """Pattern sequence item namedtuple."""
340

341
def clamp(value, min_val, max_val):
342
    """
343
    Clamp value to specified range.
344
    
345
    Parameters:
346
    - value: numeric - Value to clamp
347
    - min_val: numeric - Minimum value
348
    - max_val: numeric - Maximum value
349
    
350
    Returns:
351
    numeric - Clamped value
352
    """
353

354
def escape_chars(md, chars):
355
    """
356
    Add escape characters to Markdown instance.
357
    
358
    Parameters:
359
    - md: Markdown - Markdown instance
360
    - chars: str - Characters to escape
361
    """
362

363
def deprecated(message):
364
    """
365
    Deprecation decorator for functions.
366
    
367
    Parameters:
368
    - message: str - Deprecation message
369
    
370
    Returns:
371
    Decorator function
372
    """
373

374
def warn_deprecated(message):
375
    """
376
    Issue deprecation warning.
377
    
378
    Parameters:
379
    - message: str - Warning message
380
    """
381

382
# Platform detection utilities
383
def is_win():
384
    """Check if running on Windows."""
385

386
def is_linux():
387
    """Check if running on Linux."""  
388

389
def is_mac():
390
    """Check if running on macOS."""
391

392
# URL and path utilities
393
def url2path(url):
394
    """Convert URL to file path."""
395

396
def path2url(path):
397
    """Convert file path to URL."""
398

399
def parse_url(url):
400
    """Parse URL with file detection."""
401

402
# Unicode utilities
403
def get_code_points(text):
404
    """Get Unicode code points from text."""
405

406
def get_ord(char):
407
    """Get Unicode ordinal value."""
408

409
def get_char(code_point):
410
    """Get character from Unicode code point."""
411
```
412

413
## Advanced Configuration Patterns
414

415
### Snippet Security Configuration
416

417
```python
418
extension_configs = {
419
    'pymdownx.snippets': {
420
        'base_path': './safe_directory',
421
        'check_paths': True,
422
        'url_timeout': 5.0,
423
        'url_max_size': 512000,  # 512KB limit
424
        'url_request_headers': {
425
            'User-Agent': 'PyMdown-Extensions/1.0'
426
        }
427
    }
428
}
429
```
430

431
### HTML Sanitization Configuration
432

433
```python
434
extension_configs = {
435
    'pymdownx.striphtml': {
436
        'strip_comments': True,
437
        'strip_js_on_attributes': True,
438
        'strip_attributes': [
439
            'onclick', 'onload', 'onerror', 'onmouseover',
440
            'style'  # Remove inline styles
441
        ]
442
    }
443
}
444
```
445

446
### CriticMarkup Workflow
447

448
```python
449
# Review mode - show all changes
450
review_md = markdown.Markdown(extensions=[
451
    'pymdownx.critic'
452
], extension_configs={
453
    'pymdownx.critic': {'mode': 'view'}
454
})
455

456
# Accept changes
457
accept_md = markdown.Markdown(extensions=[
458
    'pymdownx.critic'  
459
], extension_configs={
460
    'pymdownx.critic': {'mode': 'accept'}
461
})
462

463
# Reject changes
464
reject_md = markdown.Markdown(extensions=[
465
    'pymdownx.critic'
466
], extension_configs={
467
    'pymdownx.critic': {'mode': 'reject'}
468
})
469
```
470

471
## Database and Constants
472

473
### Emoji and Key Databases
474

475
```python { .api }
476
# From emoji database modules
477
emoji1_db = {
478
    'name': 'emoji1',
479
    'emoji': {...},  # Emoji data dictionary
480
    'aliases': {...}  # Alias mappings
481
}
482

483
gemoji_db = {
484
    'name': 'gemoji', 
485
    'emoji': {...},
486
    'aliases': {...}
487
}
488

489
twemoji_db = {
490
    'name': 'twemoji',
491
    'emoji': {...},
492
    'aliases': {...}
493
}
494

495
keymap_db = {
496
    'name': 'keymap',
497
    'keymap': {...}  # Key mapping data
498
}
499
```
500

501
### Critic Constants
502

503
```python { .api }
504
CRITIC_KEY = "critic"  # Critic stash key
505
SOH = "\x01"  # Start of heading marker
506
EOT = "\x04"  # End of transmission marker
507
```
508

509
## Types
510

511
```python { .api }
512
from typing import Any, Callable, Dict, List, Optional, Union, Tuple
513
from markdown import Extension, Preprocessor, Postprocessor, InlineProcessor
514

515
# Utility types
516
DeprecationDecorator = Callable[[Callable], Callable]  # Deprecation decorator
517
Validator = Callable[[Any], bool]  # Validation function
518
Slugifier = Callable[[str, str], str]  # Slugify function
519

520
# File and URL types
521
FilePath = str  # File path string
522
URL = str  # URL string
523
Encoding = str  # Text encoding
524
Timeout = float  # Timeout in seconds
525
MaxSize = int  # Maximum size in bytes
526
Headers = Dict[str, str]  # HTTP headers
527

528
# Content processing types
529
ContentFilter = Callable[[str], str]  # Content filter function
530
HTMLProcessor = Callable[[str], str]  # HTML processing function
531
EscapeHandler = Callable[[str], str]  # Escape handling function
532

533
# Critic markup types
534
CriticMode = str  # Critic processing mode ('view', 'accept', 'reject')
535
CriticMark = str  # Critic markup string
536
```