0
# Extension System
1
2
Plugin architecture for extending ASDF with custom types, validators, compressors, and tags. Enables seamless integration with domain-specific libraries and custom data formats while maintaining compatibility with the ASDF standard.
3
4
## Capabilities
5
6
### Base Extension Class
7
8
Abstract base class for creating ASDF extensions that define custom types, validation rules, and serialization behavior.
9
10
```python { .api }
11
class Extension:
12
"""
13
Abstract base class for ASDF extensions.
14
"""
15
16
@property
17
def extension_uri(self) -> str:
18
"""
19
Unique URI identifying this extension.
20
Must be implemented by subclasses.
21
"""
22
23
@property
24
def tags(self) -> list:
25
"""
26
List of YAML tag URIs supported by this extension.
27
"""
28
29
@property
30
def converters(self) -> list:
31
"""
32
List of Converter objects for handling custom types.
33
"""
34
35
@property
36
def compressors(self) -> list:
37
"""
38
List of Compressor objects for custom compression schemes.
39
"""
40
41
@property
42
def validators(self) -> list:
43
"""
44
List of Validator objects for additional validation.
45
"""
46
```
47
48
### Extension Proxy
49
50
Wrapper that provides default implementations and manages extension lifecycle.
51
52
```python { .api }
53
class ExtensionProxy:
54
"""
55
Wrapper providing default implementations for extensions.
56
"""
57
58
def __init__(self, extension, package_name=None, package_version=None):
59
"""
60
Create extension proxy.
61
62
Parameters:
63
- extension: Extension instance to wrap
64
- package_name (str, optional): Package name for metadata
65
- package_version (str, optional): Package version for metadata
66
"""
67
```
68
69
### Converter System
70
71
Convert custom Python types to/from ASDF-serializable representations.
72
73
```python { .api }
74
class Converter:
75
"""
76
Abstract base class for type converters.
77
"""
78
79
def can_convert(self, obj) -> bool:
80
"""
81
Check if this converter can handle the given object.
82
83
Parameters:
84
- obj: Object to check
85
86
Returns:
87
bool: True if this converter can handle the object
88
"""
89
90
def convert(self, obj, **kwargs):
91
"""
92
Convert object to ASDF-serializable form.
93
94
Parameters:
95
- obj: Object to convert
96
- **kwargs: Additional conversion options
97
98
Returns:
99
ASDF-serializable representation
100
"""
101
102
def can_convert_to_tree(self, obj_type) -> bool:
103
"""
104
Check if this converter can convert objects of given type to tree form.
105
106
Parameters:
107
- obj_type: Type to check
108
109
Returns:
110
bool: True if type can be converted to tree
111
"""
112
113
def convert_to_tree(self, obj, ctx):
114
"""
115
Convert object to tree representation for YAML serialization.
116
117
Parameters:
118
- obj: Object to convert
119
- ctx: Serialization context
120
121
Returns:
122
Tree representation suitable for YAML
123
"""
124
125
def convert_from_tree(self, tree, ctx):
126
"""
127
Convert tree representation back to Python object.
128
129
Parameters:
130
- tree: Tree representation from YAML
131
- ctx: Deserialization context
132
133
Returns:
134
Reconstructed Python object
135
"""
136
137
class ConverterProxy:
138
"""
139
Wrapper for converter instances providing metadata and lifecycle management.
140
"""
141
142
def __init__(self, converter, tags, package_name=None, package_version=None):
143
"""
144
Create converter proxy.
145
146
Parameters:
147
- converter: Converter instance to wrap
148
- tags (list): YAML tags this converter handles
149
- package_name (str, optional): Package name for metadata
150
- package_version (str, optional): Package version for metadata
151
"""
152
```
153
154
### Compression System
155
156
Custom compression algorithms for array data and file content.
157
158
```python { .api }
159
class Compressor:
160
"""
161
Abstract base class for compression algorithms.
162
"""
163
164
@property
165
def label(self) -> str:
166
"""
167
Short label identifying this compression algorithm.
168
"""
169
170
def compress(self, data, **kwargs):
171
"""
172
Compress data using this algorithm.
173
174
Parameters:
175
- data (bytes): Data to compress
176
- **kwargs: Algorithm-specific options
177
178
Returns:
179
bytes: Compressed data
180
"""
181
182
def decompress(self, data, **kwargs):
183
"""
184
Decompress data using this algorithm.
185
186
Parameters:
187
- data (bytes): Compressed data
188
- **kwargs: Algorithm-specific options
189
190
Returns:
191
bytes: Decompressed data
192
"""
193
```
194
195
### Validation System
196
197
Additional schema validation beyond core ASDF schemas.
198
199
```python { .api }
200
class Validator:
201
"""
202
Abstract base class for additional validation.
203
"""
204
205
def validate(self, data, schema_uri, **kwargs):
206
"""
207
Validate data against additional constraints.
208
209
Parameters:
210
- data: Data to validate
211
- schema_uri (str): URI of schema being validated against
212
- **kwargs: Validation options
213
214
Raises:
215
ValidationError: If validation fails
216
"""
217
```
218
219
### Tag Definition
220
221
Define YAML tags for custom object serialization.
222
223
```python { .api }
224
class TagDefinition:
225
"""
226
Definition of a YAML tag for ASDF objects.
227
"""
228
229
def __init__(self, tag_uri, schema_uris=None):
230
"""
231
Create tag definition.
232
233
Parameters:
234
- tag_uri (str): URI of the YAML tag
235
- schema_uris (list, optional): URIs of associated schemas
236
"""
237
238
@property
239
def tag_uri(self) -> str:
240
"""URI of the YAML tag."""
241
242
@property
243
def schema_uris(self) -> list:
244
"""List of schema URIs associated with this tag."""
245
```
246
247
### Extension Management
248
249
System for managing collections of extensions and their interactions.
250
251
```python { .api }
252
class ExtensionManager:
253
"""
254
Manages collection of extensions and their interactions.
255
"""
256
257
def get_extensions(self) -> list:
258
"""
259
Get all managed extensions.
260
261
Returns:
262
list: All Extension objects under management
263
"""
264
265
def get_converter_for_type(self, typ):
266
"""
267
Find converter capable of handling given type.
268
269
Parameters:
270
- typ: Type to find converter for
271
272
Returns:
273
Converter: Converter capable of handling the type, or None
274
"""
275
276
def get_validator_for_uri(self, uri):
277
"""
278
Find validator for given schema URI.
279
280
Parameters:
281
- uri (str): Schema URI
282
283
Returns:
284
Validator: Validator for the URI, or None
285
"""
286
287
def get_cached_extension_manager(extensions=None):
288
"""
289
Get cached extension manager instance.
290
291
Parameters:
292
- extensions (list, optional): Additional extensions to include
293
294
Returns:
295
ExtensionManager: Cached manager instance
296
"""
297
```
298
299
### Serialization Context
300
301
Context information available during serialization and deserialization operations.
302
303
```python { .api }
304
class SerializationContext:
305
"""
306
Context information during serialization/deserialization.
307
"""
308
309
@property
310
def extension_manager(self) -> ExtensionManager:
311
"""Extension manager for this context."""
312
313
@property
314
def url_mapping(self) -> dict:
315
"""URL mapping for resolving references."""
316
317
@property
318
def block_manager(self):
319
"""Block manager for array data."""
320
```
321
322
## Usage Examples
323
324
### Creating a Custom Extension
325
326
```python
327
from asdf.extension import Extension, Converter
328
import numpy as np
329
330
class ComplexNumber:
331
"""Custom complex number class with metadata."""
332
def __init__(self, real, imag, precision="double"):
333
self.real = real
334
self.imag = imag
335
self.precision = precision
336
337
class ComplexConverter(Converter):
338
"""Converter for ComplexNumber objects."""
339
340
def can_convert(self, obj):
341
return isinstance(obj, ComplexNumber)
342
343
def convert_to_tree(self, obj, ctx):
344
return {
345
'real': obj.real,
346
'imag': obj.imag,
347
'precision': obj.precision
348
}
349
350
def convert_from_tree(self, tree, ctx):
351
return ComplexNumber(
352
tree['real'],
353
tree['imag'],
354
tree.get('precision', 'double')
355
)
356
357
class ComplexExtension(Extension):
358
"""Extension for complex number support."""
359
360
extension_uri = "asdf://example.com/complex/extensions/complex-1.0.0"
361
converters = [ComplexConverter()]
362
tags = ["asdf://example.com/complex/tags/complex-1.0.0"]
363
364
# Use the extension
365
complex_num = ComplexNumber(3.0, 4.0, "single")
366
data = {"my_complex": complex_num}
367
368
af = asdf.AsdfFile(data, extensions=[ComplexExtension()])
369
af.write_to("complex_data.asdf")
370
371
# Read with extension
372
with asdf.open("complex_data.asdf", extensions=[ComplexExtension()]) as af:
373
restored = af.tree["my_complex"]
374
print(f"{restored.real} + {restored.imag}i")
375
```
376
377
### Custom Compression
378
379
```python
380
from asdf.extension import Compressor
381
import zlib
382
383
class CustomCompressor(Compressor):
384
"""Custom compression using high compression ratio."""
385
386
label = "custom_zlib"
387
388
def compress(self, data, level=9, **kwargs):
389
return zlib.compress(data, level=level)
390
391
def decompress(self, data, **kwargs):
392
return zlib.decompress(data)
393
394
class CompressionExtension(Extension):
395
"""Extension providing custom compression."""
396
397
extension_uri = "asdf://example.com/compression/extensions/custom-1.0.0"
398
compressors = [CustomCompressor()]
399
400
# Use custom compression
401
import numpy as np
402
403
data = {"large_array": np.random.random(100000)}
404
af = asdf.AsdfFile(data, extensions=[CompressionExtension()])
405
af.write_to("compressed.asdf", all_array_compression="custom_zlib")
406
```
407
408
### Validation Extension
409
410
```python
411
from asdf.extension import Validator, ValidationError
412
413
class RangeValidator(Validator):
414
"""Validates that numeric values are within specified ranges."""
415
416
def validate(self, data, schema_uri, **kwargs):
417
if "range-check" in schema_uri:
418
if isinstance(data, (int, float)):
419
if not (0 <= data <= 100):
420
raise ValidationError(f"Value {data} outside range [0, 100]")
421
422
class ValidationExtension(Extension):
423
"""Extension providing range validation."""
424
425
extension_uri = "asdf://example.com/validation/extensions/range-1.0.0"
426
validators = [RangeValidator()]
427
428
# Use validation
429
data = {"percentage": 85} # Valid
430
af = asdf.AsdfFile(data, extensions=[ValidationExtension()])
431
432
# This would raise ValidationError:
433
# data = {"percentage": 150} # Invalid
434
```
435
436
### Multi-Component Extension
437
438
```python
439
from asdf.extension import Extension, Converter, Compressor, Validator
440
import json
441
import gzip
442
443
class JsonConverter(Converter):
444
"""Converter for JSON-serializable objects."""
445
446
def can_convert(self, obj):
447
try:
448
json.dumps(obj)
449
return True
450
except (TypeError, ValueError):
451
return False
452
453
def convert_to_tree(self, obj, ctx):
454
return {"json_data": json.dumps(obj)}
455
456
def convert_from_tree(self, tree, ctx):
457
return json.loads(tree["json_data"])
458
459
class GzipCompressor(Compressor):
460
"""Gzip compression for text data."""
461
462
label = "gzip"
463
464
def compress(self, data, **kwargs):
465
return gzip.compress(data)
466
467
def decompress(self, data, **kwargs):
468
return gzip.decompress(data)
469
470
class JsonSizeValidator(Validator):
471
"""Validates JSON data size limits."""
472
473
def validate(self, data, schema_uri, **kwargs):
474
if "json-size" in schema_uri:
475
json_str = json.dumps(data)
476
if len(json_str) > 1000000: # 1MB limit
477
raise ValidationError("JSON data exceeds size limit")
478
479
class FullExtension(Extension):
480
"""Complete extension with converter, compressor, and validator."""
481
482
extension_uri = "asdf://example.com/full/extensions/full-1.0.0"
483
converters = [JsonConverter()]
484
compressors = [GzipCompressor()]
485
validators = [JsonSizeValidator()]
486
tags = ["asdf://example.com/full/tags/json-1.0.0"]
487
488
# Use complete extension
489
complex_data = {
490
"metadata": {"type": "experiment", "version": 1},
491
"parameters": [{"name": "temp", "value": 25.0}],
492
"results": list(range(1000))
493
}
494
495
af = asdf.AsdfFile(
496
{"experiment": complex_data},
497
extensions=[FullExtension()]
498
)
499
af.write_to("full_extension_example.asdf")
500
```
501
502
### Extension Discovery and Management
503
504
```python
505
# Get all available extensions
506
manager = asdf.get_cached_extension_manager()
507
extensions = manager.get_extensions()
508
509
print(f"Found {len(extensions)} extensions:")
510
for ext in extensions:
511
print(f" {ext.extension_uri}")
512
print(f" Converters: {len(ext.converters)}")
513
print(f" Compressors: {len(ext.compressors)}")
514
print(f" Validators: {len(ext.validators)}")
515
516
# Find converter for specific type
517
converter = manager.get_converter_for_type(ComplexNumber)
518
if converter:
519
print(f"Found converter for ComplexNumber: {converter}")
520
```