0
# Configuration Dictionaries
1
2
Core ConfigDict and FrozenConfigDict classes that provide the foundation of ML Collections. These classes offer dict-like structures with ML-specific enhancements including dot-based access, type safety, locking mechanisms, and serialization capabilities designed for machine learning experiment configurations.
3
4
## Capabilities
5
6
### ConfigDict Creation and Basic Operations
7
8
Create mutable configuration dictionaries with optional type safety and initialization from existing dictionaries or keyword arguments.
9
10
```python { .api }
11
class ConfigDict:
12
def __init__(
13
self,
14
initial_dictionary=None,
15
type_safe=True,
16
convert_dict=True,
17
*,
18
allow_dotted_keys=False,
19
sort_keys=True
20
):
21
"""
22
Create a new ConfigDict.
23
24
Args:
25
initial_dictionary (dict, optional): Initial values to populate the ConfigDict
26
type_safe (bool): Enable type safety enforcement (default: True)
27
convert_dict (bool): Convert nested dicts to ConfigDict (default: True)
28
allow_dotted_keys (bool): Allow keys with dots (default: False)
29
sort_keys (bool): Sort keys in string representation (default: True)
30
"""
31
32
@property
33
def is_type_safe(self) -> bool:
34
"""Returns True if type safe."""
35
36
@property
37
def allow_dotted_keys(self) -> bool:
38
"""Returns True if keys can contain dots."""
39
40
@property
41
def convert_dict(self) -> bool:
42
"""Returns True if auto-converting dicts to ConfigDict."""
43
44
@property
45
def is_locked(self) -> bool:
46
"""Returns True if object is locked."""
47
48
def create(**kwargs) -> ConfigDict:
49
"""
50
Factory function to create ConfigDict with keyword arguments.
51
52
Args:
53
**kwargs: Key-value pairs to initialize the ConfigDict
54
55
Returns:
56
ConfigDict: New ConfigDict instance with provided values
57
"""
58
```
59
60
Usage example:
61
62
```python
63
from ml_collections import ConfigDict
64
from ml_collections.config_dict import create
65
66
# Create empty ConfigDict
67
config = ConfigDict()
68
69
# Create from dictionary
70
config = ConfigDict({'learning_rate': 0.001, 'batch_size': 32})
71
72
# Create using factory function
73
config = create(
74
model='resnet50',
75
optimizer='adam',
76
learning_rate=0.001
77
)
78
```
79
80
### Field Access and Modification
81
82
Access and modify configuration fields using both dot notation and dictionary-style access with automatic type checking.
83
84
```python { .api }
85
# ConfigDict supports both dot notation and dictionary access
86
config.field_name = value # Dot notation assignment
87
value = config.field_name # Dot notation access
88
config['field_name'] = value # Dictionary-style assignment
89
value = config['field_name'] # Dictionary-style access
90
```
91
92
### Locking and Protection Mechanisms
93
94
Prevent accidental configuration changes during execution while maintaining the ability to modify existing field values.
95
96
```python { .api }
97
def lock(self) -> ConfigDict:
98
"""
99
Lock the ConfigDict to prevent addition or deletion of fields.
100
Existing field values can still be modified.
101
102
Returns:
103
ConfigDict: Returns self for method chaining
104
"""
105
106
def unlock(self) -> None:
107
"""
108
Unlock the ConfigDict to allow addition and deletion of fields.
109
"""
110
111
def unlocked(self):
112
"""
113
Context manager for temporarily unlocking the ConfigDict.
114
115
Returns:
116
Context manager that unlocks on entry and re-locks on exit
117
"""
118
```
119
120
Usage example:
121
122
```python
123
config = ConfigDict({'learning_rate': 0.001})
124
config.lock()
125
126
# This works - modifying existing field
127
config.learning_rate = 0.01
128
129
# This raises MutabilityError - adding new field to locked ConfigDict
130
try:
131
config.new_field = 'value'
132
except MutabilityError:
133
print("Cannot add new fields to locked ConfigDict")
134
135
# Temporarily unlock for modifications
136
with config.unlocked():
137
config.new_field = 'value' # This works
138
```
139
140
### Reference System Integration
141
142
Get references to fields for creating dynamic relationships and lazy computation chains.
143
144
```python { .api }
145
def get_ref(self, key: str) -> FieldReference:
146
"""
147
Get a FieldReference to a field for lazy computation.
148
149
Args:
150
key (str): Field name to reference
151
152
Returns:
153
FieldReference: Reference object for the field
154
"""
155
156
def get_oneway_ref(self, key: str) -> FieldReference:
157
"""
158
Get a one-way FieldReference that doesn't create bidirectional dependency.
159
160
Args:
161
key (str): Field name to reference
162
163
Returns:
164
FieldReference: One-way reference object for the field
165
"""
166
```
167
168
### Dictionary Operations and Iteration
169
170
Standard dictionary operations with ConfigDict-specific enhancements and iteration methods.
171
172
```python { .api }
173
def get(self, key: str, default=None):
174
"""
175
Standard dict get with default value.
176
177
Args:
178
key (str): Key to retrieve
179
default: Default value if key not found
180
181
Returns:
182
Value at key or default
183
"""
184
185
def keys(self):
186
"""Returns sorted list of keys."""
187
188
def values(self, preserve_field_references: bool = False):
189
"""
190
Returns list of values.
191
192
Args:
193
preserve_field_references (bool): Whether to preserve FieldReference objects
194
195
Returns:
196
list: List of values
197
"""
198
199
def items(self, preserve_field_references: bool = False):
200
"""
201
Returns key-value pairs.
202
203
Args:
204
preserve_field_references (bool): Whether to preserve FieldReference objects
205
206
Returns:
207
list: List of (key, value) tuples
208
"""
209
210
def iterkeys(self):
211
"""Iterator over keys."""
212
213
def itervalues(self, preserve_field_references: bool = False):
214
"""
215
Iterator over values.
216
217
Args:
218
preserve_field_references (bool): Whether to preserve FieldReference objects
219
"""
220
221
def iteritems(self, preserve_field_references: bool = False):
222
"""
223
Iterator over key-value pairs.
224
225
Args:
226
preserve_field_references (bool): Whether to preserve FieldReference objects
227
"""
228
```
229
230
### Type Management
231
232
Methods for type safety and type information retrieval.
233
234
```python { .api }
235
def get_type(self, key: str):
236
"""
237
Returns type of field associated with key.
238
239
Args:
240
key (str): Field name
241
242
Returns:
243
type: Type of the field
244
"""
245
246
def ignore_type(self):
247
"""
248
Context manager to temporarily disable type safety.
249
250
Returns:
251
Context manager that disables type checking on entry and re-enables on exit
252
"""
253
```
254
255
### Update Operations
256
257
Advanced update methods for bulk configuration changes.
258
259
```python { .api }
260
def update(self, *other, **kwargs):
261
"""
262
Recursive update from dict-like objects.
263
264
Args:
265
*other: Dict-like objects to update from
266
**kwargs: Additional key-value pairs
267
"""
268
269
def update_from_flattened_dict(self, flattened_dict: dict, strip_prefix: str = ''):
270
"""
271
Update from flattened dictionary with dot-separated keys.
272
273
Args:
274
flattened_dict (dict): Dictionary with dot-separated keys
275
strip_prefix (str): Prefix to strip from keys
276
"""
277
```
278
279
### Serialization Methods
280
281
Convert ConfigDict to various serialization formats.
282
283
```python { .api }
284
def to_yaml(self, **kwargs) -> str:
285
"""
286
Returns YAML representation.
287
288
Args:
289
**kwargs: Arguments passed to yaml.dump
290
291
Returns:
292
str: YAML string representation
293
"""
294
295
def to_json(self, json_encoder_cls=None, **kwargs) -> str:
296
"""
297
Returns JSON representation.
298
299
Args:
300
json_encoder_cls: Custom JSON encoder class
301
**kwargs: Arguments passed to json.dumps
302
303
Returns:
304
str: JSON string representation
305
"""
306
307
def to_json_best_effort(self, **kwargs) -> str:
308
"""
309
Best effort JSON serialization handling non-serializable objects.
310
311
Args:
312
**kwargs: Arguments passed to json.dumps
313
314
Returns:
315
str: JSON string representation
316
"""
317
318
def to_dict(self, visit_map=None, preserve_field_references: bool = False) -> dict:
319
"""
320
Convert to regular dictionary.
321
322
Args:
323
visit_map: Mapping to track visited objects (for cycle detection)
324
preserve_field_references (bool): Whether to preserve FieldReference objects
325
326
Returns:
327
dict: Regular Python dictionary
328
"""
329
```
330
331
### ConfigDict Utilities and Conversion
332
333
Utility methods for copying, comparison, and resolving references within ConfigDict structures.
334
335
```python { .api }
336
def copy_and_resolve_references(self) -> ConfigDict:
337
"""
338
Create a copy with all FieldReferences resolved to actual values.
339
340
Returns:
341
ConfigDict: New ConfigDict with resolved references
342
"""
343
344
def eq_as_configdict(self, other) -> bool:
345
"""
346
Equality comparison treating both objects as ConfigDict.
347
348
Args:
349
other: Object to compare with
350
351
Returns:
352
bool: True if objects are equal as ConfigDict
353
"""
354
```
355
356
### FrozenConfigDict - Immutable Configurations
357
358
Create immutable, hashable versions of configurations for reproducibility and caching in ML workflows.
359
360
```python { .api }
361
class FrozenConfigDict:
362
def __init__(self, initial_dictionary=None):
363
"""
364
Create an immutable, hashable ConfigDict.
365
366
Args:
367
initial_dictionary (dict or ConfigDict): Initial values
368
"""
369
370
def as_configdict(self) -> ConfigDict:
371
"""
372
Convert FrozenConfigDict back to mutable ConfigDict.
373
374
Returns:
375
ConfigDict: Mutable copy of this FrozenConfigDict
376
"""
377
378
def eq_as_configdict(self, other) -> bool:
379
"""
380
Equality comparison treating both objects as ConfigDict.
381
382
Args:
383
other: Object to compare with
384
385
Returns:
386
bool: True if objects are equal as ConfigDict
387
"""
388
```
389
390
Usage example:
391
392
```python
393
from ml_collections import ConfigDict, FrozenConfigDict
394
395
# Create mutable config
396
config = ConfigDict({
397
'model': 'resnet50',
398
'learning_rate': 0.001,
399
'optimizer': {'name': 'adam', 'beta1': 0.9}
400
})
401
402
# Create immutable version
403
frozen_config = FrozenConfigDict(config)
404
405
# Frozen configs are hashable and can be used as dictionary keys
406
model_cache = {frozen_config: trained_model}
407
408
# Convert back to mutable when needed
409
mutable_copy = frozen_config.as_configdict()
410
mutable_copy.learning_rate = 0.01 # This works
411
412
# Original frozen config is unchanged
413
print(frozen_config.learning_rate) # Still 0.001
414
```
415
416
### Recursive Field Operations
417
418
Perform bulk operations on nested configuration structures.
419
420
```python { .api }
421
def recursive_rename(config, rename_fn):
422
"""
423
Recursively rename fields in configuration structures.
424
425
Args:
426
config: ConfigDict or FrozenConfigDict to rename fields in
427
rename_fn: Function that takes old field name and returns new name
428
429
Returns:
430
Same type as input config with renamed fields
431
"""
432
```
433
434
## Types
435
436
```python { .api }
437
class MutabilityError(AttributeError):
438
"""Raised when attempting to modify locked or immutable structures."""
439
440
class JSONDecodeError(ValueError):
441
"""Raised when JSON decoding fails."""
442
443
class CustomJSONEncoder:
444
"""JSON encoder for ML Collections objects."""
445
```