or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

config-dict.mdconfig-flags.mdfield-references.mdindex.md
tile.json

tessl/pypi-ml-collections

ML Collections is a library of Python collections designed for ML usecases.

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/ml-collections@1.1.x

To install, run

npx @tessl/cli install tessl/pypi-ml-collections@1.1.0

index.mddocs/

ML Collections

ML Collections is a library of Python collections designed specifically for machine learning use cases. It provides specialized configuration dictionaries with dot-based access, type safety, locking mechanisms, lazy computation, and human-readable YAML serialization - essential features for managing complex ML experiment configurations and model parameters.

Package Information

  • Package Name: ml-collections
  • Language: Python
  • Installation: pip install ml-collections
  • Python Requirement: >=3.10

Core Imports

import ml_collections
from ml_collections import ConfigDict, FieldReference, FrozenConfigDict

Individual imports:

from ml_collections.config_dict import (
    ConfigDict, FrozenConfigDict, FieldReference,
    create, placeholder, required_placeholder, recursive_rename,
    MutabilityError, RequiredValueError, JSONDecodeError, CustomJSONEncoder
)
from ml_collections.config_flags import (
    DEFINE_config_file, DEFINE_config_dict, DEFINE_config_dataclass,
    get_config_filename, get_override_values, is_config_flag,
    register_flag_parser, register_flag_parser_for_type,
    UnsupportedOperationError, FlagOrderError, UnparsedFlagError
)

Basic Usage

from ml_collections import ConfigDict, FieldReference, FrozenConfigDict

# Create a mutable configuration dictionary
config = ConfigDict()
config.model = 'resnet50'
config.learning_rate = 0.001
config.batch_size = 32

# Nested configurations with dot access
config.optimizer = ConfigDict()
config.optimizer.name = 'adam'
config.optimizer.beta1 = 0.9
config.optimizer.beta2 = 0.999

# Access fields with dot notation
print(config.learning_rate)  # 0.001
print(config.optimizer.name)  # 'adam'

# Create immutable configuration for reproducibility
frozen_config = FrozenConfigDict(config)
print(hash(frozen_config))  # Hashable for caching

# Field references for dynamic linking
config.steps_per_epoch = 1000
config.epochs = 10
config.total_steps = config.get_ref('epochs') * config.get_ref('steps_per_epoch')
print(config.total_steps)  # 10000

Architecture

ML Collections follows a layered architecture:

  • ConfigDict/FrozenConfigDict: Core dictionary classes with ML-specific enhancements
  • FieldReference System: Lazy computation and dynamic field linking
  • Type Safety Layer: Runtime type checking and validation
  • Locking Mechanism: Configuration structure protection
  • Serialization Layer: YAML-compatible human-readable output
  • Command-line Integration: absl.flags integration for experiment management

This design enables reproducible ML experiments while maintaining flexibility for complex parameter configurations and nested model settings.

Capabilities

Configuration Dictionaries

Core ConfigDict and FrozenConfigDict classes providing dot-based access, type safety, locking mechanisms, and ML-specific features for managing experiment configurations.

class ConfigDict:
    def __init__(self, initial_dictionary=None, type_safe=True): ...
    def lock(self) -> None: ...
    def unlock(self) -> None: ...
    def get_ref(self, key: str) -> FieldReference: ...

class FrozenConfigDict:
    def __init__(self, initial_dictionary=None): ...
    def as_configdict(self) -> ConfigDict: ...

Configuration Dictionaries

Field References and Lazy Computation

Dynamic field linking system enabling lazy computation, parameter dependencies, and flexible configuration relationships for complex ML experiment setups.

class FieldReference:
    def get(self): ...
    def set(self, value): ...
    def copy(self) -> FieldReference: ...

def placeholder(fn_or_value=None, *, required=False): ...
def required_placeholder(): ...

Field References

Command-line Configuration Flags

Integration with absl.flags for loading configurations from files, defining command-line overrides, and managing parameterized experiment configurations.

def DEFINE_config_file(
    name: str,
    default: Optional[str] = None,
    help_string: str = "path to config file.",
    **kwargs
): ...

def DEFINE_config_dict(
    name: str, 
    config: ConfigDict,
    help_string: str = "ConfigDict instance.",
    **kwargs
): ...

Configuration Flags

Types

class RequiredValueError(ValueError):
    """Raised when a required placeholder value is not provided."""

class MutabilityError(AttributeError):
    """Raised when attempting to modify locked or immutable structures."""

class JSONDecodeError(ValueError):
    """Raised when JSON decoding fails."""

class CustomJSONEncoder:
    """JSON encoder for ML Collections objects."""

class UnsupportedOperationError(Exception):
    """Raised for unsupported flag operations."""

class FlagOrderError(ValueError):
    """Raised when flags are accessed in wrong order."""

class UnparsedFlagError(ValueError):
    """Raised when flags haven't been parsed."""