0
# Container System
1
2
HDMF's container system provides the foundation for organizing hierarchical data structures with metadata, parent-child relationships, and data management capabilities. The system is built around abstract and concrete container classes that enable flexible data organization and manipulation.
3
4
## Capabilities
5
6
### Abstract Container Base
7
8
The foundational abstract base class for all container objects in HDMF, providing core functionality for hierarchical data organization.
9
10
```python { .api }
11
class AbstractContainer:
12
"""
13
Abstract base class for all container objects.
14
15
Properties:
16
- name (str): Name of the container
17
- parent (AbstractContainer): Parent container
18
- children (tuple): Child containers
19
- fields (dict): Field values
20
- object_id (str): Unique object identifier
21
- modified (bool): Whether container has been modified
22
- container_source (str): Source information
23
"""
24
25
def __init__(self, name: str):
26
"""
27
Initialize container with name.
28
29
Args:
30
name: Name for the container
31
"""
32
33
def get_ancestor(self, neurodata_type: str = None) -> 'AbstractContainer':
34
"""
35
Get ancestor container of specified type.
36
37
Args:
38
neurodata_type: Type of ancestor to find
39
40
Returns:
41
Ancestor container or None
42
"""
43
44
def all_children(self) -> Generator['AbstractContainer', None, None]:
45
"""
46
Get generator for all child containers recursively.
47
48
Returns:
49
Generator yielding child containers
50
"""
51
52
def generate_new_id(self) -> str:
53
"""
54
Generate new unique object ID.
55
56
Returns:
57
New unique identifier string
58
"""
59
60
def set_modified(self, modified: bool = True):
61
"""
62
Mark container as modified or unmodified.
63
64
Args:
65
modified: Whether container is modified
66
"""
67
68
def reset_parent(self):
69
"""Reset parent relationship for this container."""
70
71
@property
72
def name(self) -> str:
73
"""Name of the container."""
74
75
@property
76
def parent(self) -> 'AbstractContainer':
77
"""Parent container."""
78
79
@property
80
def children(self) -> tuple:
81
"""Tuple of child containers."""
82
```
83
84
### Container Class
85
86
Concrete container class extending AbstractContainer with enhanced display and I/O functionality.
87
88
```python { .api }
89
class Container(AbstractContainer):
90
"""
91
Container that can hold other containers with enhanced printing/HTML functionality.
92
"""
93
94
def __init__(self, name: str):
95
"""
96
Initialize container.
97
98
Args:
99
name: Name for the container
100
"""
101
102
def set_data_io(self, data_io):
103
"""
104
Set data I/O object for this container.
105
106
Args:
107
data_io: Data I/O configuration object
108
"""
109
110
def _repr_html_(self) -> str:
111
"""
112
Generate HTML representation for Jupyter notebooks.
113
114
Returns:
115
HTML string representation
116
"""
117
```
118
119
### Data Container Class
120
121
Specialized container for holding dataset information with data manipulation capabilities.
122
123
```python { .api }
124
class Data(AbstractContainer):
125
"""
126
Container class for representing datasets with data manipulation capabilities.
127
128
Properties:
129
- data: The actual data content
130
- shape: Shape of the data array
131
"""
132
133
def __init__(self, name: str, data):
134
"""
135
Initialize data container.
136
137
Args:
138
name: Name for the data container
139
data: Data content (array-like)
140
"""
141
142
def get(self):
143
"""
144
Get the data content.
145
146
Returns:
147
The data content
148
"""
149
150
def append(self, arg):
151
"""
152
Append data to existing data.
153
154
Args:
155
arg: Data to append
156
"""
157
158
def extend(self, arg):
159
"""
160
Extend data with iterable.
161
162
Args:
163
arg: Iterable to extend data with
164
"""
165
166
def transform(self, transform_func):
167
"""
168
Apply transformation function to data.
169
170
Args:
171
transform_func: Function to transform data
172
"""
173
174
def set_data_io(self, data_io):
175
"""
176
Set data I/O configuration.
177
178
Args:
179
data_io: Data I/O configuration object
180
"""
181
182
@property
183
def data(self):
184
"""The data content."""
185
186
@property
187
def shape(self) -> tuple:
188
"""Shape of the data array."""
189
```
190
191
### Multi-Container Interface
192
193
Dynamic container class for managing multiple containers of the same type with auto-generated methods.
194
195
```python { .api }
196
class MultiContainerInterface:
197
"""
198
Base class for containers that hold multiple containers of the same type.
199
200
Features:
201
- Auto-generates methods based on __clsconf__ attribute
202
- Provides dictionary-like access to child containers
203
- Supports iteration over contained items
204
"""
205
206
def __init__(self, **kwargs):
207
"""Initialize multi-container interface."""
208
209
def __getitem__(self, key):
210
"""Get container by key."""
211
212
def __iter__(self):
213
"""Iterate over contained items."""
214
```
215
216
### External Resource Manager
217
218
Manager for handling external resources and references (HERD - Hierarchical External Resource Descriptor).
219
220
```python { .api }
221
class HERDManager:
222
"""
223
Manager for external resources using HERD (Hierarchical External Resource Descriptor).
224
"""
225
226
def __init__(self):
227
"""Initialize HERD manager."""
228
229
def link_resources(self, container: Container, resources: dict):
230
"""
231
Link external resources to a container.
232
233
Args:
234
container: Container to link resources to
235
resources: Dictionary of resource specifications
236
"""
237
238
def get_linked_resources(self, container: Container) -> dict:
239
"""
240
Get linked resources for a container.
241
242
Args:
243
container: Container to get resources for
244
245
Returns:
246
Dictionary of linked resources
247
"""
248
```
249
250
### Table System
251
252
Specialized containers for tabular data structures with row-based access and DataFrame integration.
253
254
```python { .api }
255
class Table(Data):
256
"""
257
Base table implementation with column support and row-based access.
258
"""
259
260
def __init__(self, name: str, description: str, **kwargs):
261
"""
262
Initialize table.
263
264
Args:
265
name: Name of the table
266
description: Description of the table
267
"""
268
269
def add_row(self, **kwargs):
270
"""
271
Add a row to the table.
272
273
Args:
274
**kwargs: Column values for the new row
275
"""
276
277
def which(self, **kwargs) -> list:
278
"""
279
Find rows matching criteria.
280
281
Args:
282
**kwargs: Search criteria
283
284
Returns:
285
List of matching row indices
286
"""
287
288
def to_dataframe(self):
289
"""
290
Convert table to pandas DataFrame.
291
292
Returns:
293
pandas.DataFrame representation
294
"""
295
296
@classmethod
297
def from_dataframe(cls, df, name: str, **kwargs):
298
"""
299
Create table from pandas DataFrame.
300
301
Args:
302
df: pandas DataFrame
303
name: Name for the table
304
305
Returns:
306
New Table instance
307
"""
308
309
class Row:
310
"""
311
Represents a row from a Table object.
312
313
Properties:
314
- idx: Row index
315
- table: Parent table
316
"""
317
318
def __init__(self, table: Table, idx: int):
319
"""
320
Initialize row.
321
322
Args:
323
table: Parent table
324
idx: Row index
325
"""
326
327
def todict(self) -> dict:
328
"""
329
Convert row to dictionary.
330
331
Returns:
332
Dictionary representation of row
333
"""
334
335
class RowGetter:
336
"""
337
Helper class providing __getitem__ functionality for Table rows.
338
"""
339
340
def __init__(self, table: Table):
341
"""
342
Initialize row getter.
343
344
Args:
345
table: Parent table
346
"""
347
348
def __getitem__(self, key):
349
"""Get row(s) by index or slice."""
350
```
351
352
## Usage Examples
353
354
### Basic Container Usage
355
356
```python
357
from hdmf import Container, Data
358
import numpy as np
359
360
# Create data container
361
data = np.array([[1, 2, 3], [4, 5, 6]])
362
data_container = Data(name='sample_data', data=data)
363
364
# Create parent container
365
parent = Container(name='experiment')
366
parent.add_child(data_container)
367
368
# Access data
369
print(f"Data shape: {data_container.shape}")
370
print(f"Data content: {data_container.get()}")
371
```
372
373
### Table Operations
374
375
```python
376
from hdmf.container import Table
377
import pandas as pd
378
379
# Create table from DataFrame
380
df = pd.DataFrame({
381
'id': [1, 2, 3],
382
'name': ['Alice', 'Bob', 'Charlie'],
383
'score': [95.5, 87.2, 91.8]
384
})
385
386
table = Table.from_dataframe(df, name='results', description='Test results')
387
388
# Add new row
389
table.add_row(id=4, name='Diana', score=88.9)
390
391
# Query rows
392
high_scores = table.which(score__gt=90)
393
print(f"High scorers: {high_scores}")
394
395
# Convert back to DataFrame
396
result_df = table.to_dataframe()
397
```
398
399
### Multi-Container Management
400
401
```python
402
from hdmf import Container, MultiContainerInterface
403
404
class ExperimentContainer(Container, MultiContainerInterface):
405
__clsconf__ = {
406
'attr': 'trials',
407
'type': Container,
408
'add': 'add_trial',
409
'get': 'get_trial'
410
}
411
412
def __init__(self, name):
413
super().__init__(name=name)
414
self.trials = []
415
416
# Usage
417
experiment = ExperimentContainer(name='behavioral_experiment')
418
trial1 = Container(name='trial_001')
419
experiment.add_trial(trial1)
420
421
# Access trial
422
retrieved_trial = experiment.get_trial('trial_001')
423
```