or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

configuration.mdcore-data-types.mddata-serialization.mdextension-system.mdfile-operations.mdindex.mdutilities.md
tile.json

tessl/pypi-asdf

Python implementation of the Advanced Scientific Data Format (ASDF) Standard

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/asdf@4.5.x

To install, run

npx @tessl/cli install tessl/pypi-asdf@4.5.0

index.mddocs/

ASDF - Advanced Scientific Data Format

ASDF is a next-generation interchange format for scientific data. This Python implementation provides hierarchical and human-readable metadata in YAML format, efficient binary array storage with memory mapping and flexible compression, content validation using JSON Schema, and native support for most Python data types with an extensible API for custom objects.

Package Information

  • Package Name: asdf
  • Language: Python
  • Installation: pip install asdf
  • Requires Python: >=3.9

Core Imports

import asdf

Common patterns:

from asdf import AsdfFile, open, load, dump

Basic Usage

import asdf
import numpy as np

# Create some data
sequence = np.arange(100)
squares = sequence**2
random = np.random.random(100)

# Store the data in an arbitrarily nested dictionary
tree = {
    "foo": 42,
    "name": "Monty",
    "sequence": sequence,
    "powers": {"squares": squares},
    "random": random,
}

# Create and write ASDF file
af = asdf.AsdfFile(tree)
af.write_to("example.asdf")

# Read the file back
with asdf.open("example.asdf") as af:
    print(af.tree["foo"])  # 42
    print(af.tree["sequence"])  # numpy array

Architecture

ASDF follows a layered architecture:

  • File Format: YAML header with binary array blocks
  • AsdfFile: Main file object managing tree data and metadata
  • Extensions: Plugin system for custom types and validation
  • Block Manager: Handles binary array storage and compression
  • Schema Validation: JSON Schema-based validation system
  • Tree Structure: Hierarchical data organization with references

Capabilities

File Operations

Core file handling functionality for creating, reading, writing, and managing ASDF files. Includes both high-level convenience functions and low-level file management.

class AsdfFile:
    def __init__(self, tree=None, uri=None, extensions=None, version=None, 
                 ignore_unrecognized_tag=False, memmap=False, lazy_load=True, 
                 custom_schema=None): ...

def open(fd, uri=None, mode=None, validate_checksums=False, extensions=None, 
         ignore_unrecognized_tag=False, _force_raw_types=False, memmap=False, 
         lazy_tree=NotSet, lazy_load=True, custom_schema=None, 
         strict_extension_check=False, ignore_missing_extensions=False): ...

def load(fp, *, uri=None, validate_checksums=False, extensions=None, 
         custom_schema=None): ...

def dump(tree, fp, *, version=None, extensions=None, all_array_storage=NotSet, 
         all_array_compression=NotSet, compression_kwargs=NotSet, pad_blocks=False, 
         custom_schema=None): ...

File Operations

Data Serialization

High-level functions for serializing and deserializing Python objects to/from ASDF format, providing convenient string-based operations alongside file-based operations.

def loads(asdf_string, *, uri=None, validate_checksums=False, extensions=None, 
          custom_schema=None): ...

def dumps(tree, *, version=None, extensions=None, all_array_storage=NotSet, 
          all_array_compression=NotSet, compression_kwargs=NotSet, pad_blocks=False, 
          custom_schema=None) -> str: ...

Data Serialization

Configuration Management

Global and context-specific configuration for controlling ASDF behavior including validation, version defaults, array handling, and I/O settings.

def get_config(): ...
def config_context(): ...

class AsdfConfig:
    validate_on_read: bool
    default_version: str
    io_block_size: int
    array_inline_threshold: int
    all_array_storage: str
    all_array_compression: str
    all_array_compression_kwargs: dict
    legacy_fill_schema_defaults: bool
    default_array_save_base: bool
    convert_unknown_ndarray_subclasses: bool
    lazy_tree: bool
    resource_mappings: list  # read-only
    resource_manager: object  # read-only
    extensions: list  # read-only

Configuration Management

Core Data Types

Built-in ASDF data types for handling specialized scientific data including large integers, external array references, and streaming arrays.

class IntegerType:
    def __init__(self, value, storage_type="internal"): ...

class ExternalArrayReference:
    def __init__(self, fileuri, target, dtype, shape): ...

class Stream:
    def __init__(self, shape, dtype, strides=None): ...

Core Data Types

Extension System

Plugin architecture for extending ASDF with custom types, validators, compressors, and tags. Enables integration with domain-specific libraries and custom data formats.

class Extension:
    extension_uri: str
    tags: List[str]
    converters: List[Converter]
    compressors: List[Compressor]
    validators: List[Validator]

class Converter:
    def can_convert(self, obj): ...
    def convert(self, obj, **kwargs): ...

class ExtensionManager:
    def get_extensions(self): ...
    def get_converter_for_type(self, typ): ...

Extension System

Utilities and Helpers

Utility functions for inspecting ASDF files, testing extensions, and working with ASDF data structures programmatically.

def info(node_or_path, max_rows=24, max_cols=120, show_values=True): ...

Utilities and Helpers

Types

class ValidationError(Exception):
    """Raised when ASDF validation fails."""

class AsdfObject(dict):
    """Dict-like container for ASDF objects with special YAML serialization."""

class Software(dict):
    """Metadata about software used to create ASDF file."""

class HistoryEntry(dict):
    """Single entry in ASDF file history."""

__version__: str
    """Current version of the ASDF library."""