0
# Data Management
1

2
Adapter functionality for managing collections of models, loading data from various sources, and providing query and storage operations through configurable storage backends.
3

4
## Capabilities
5

6
### Adapter Class
7

8
Container class for storing groups of DiffSyncModel instances and performing diff/sync operations between different data sources.
9

10
```python { .api }
11
class Adapter:
12
    """Class for storing a group of DiffSyncModel instances and diffing/synchronizing to another Adapter instance."""
13
    
14
    type: Optional[str] = None
15
    top_level: ClassVar[List[str]] = []
16
    
17
    def __init__(self, name: Optional[str] = None,
18
                 internal_storage_engine: Union[Type[BaseStore], BaseStore] = LocalStore) -> None:
19
        """
20
        Generic initialization function.
21
        
22
        Args:
23
            name: Optional name for this adapter instance
24
            internal_storage_engine: Storage backend class or instance to use
25
        """
26
```
27

28
#### Usage Example
29

30
```python
31
from diffsync import Adapter, DiffSyncModel, LocalStore
32

33
class Device(DiffSyncModel):
34
    _modelname = "device"
35
    _identifiers = ("name",)
36
    _attributes = ("vendor", "model")
37
    
38
    name: str
39
    vendor: str
40
    model: str
41

42
class NetworkAdapter(Adapter):
43
    device = Device  # Map model name to class
44
    top_level = ["device"]  # Top-level models for traversal
45
    
46
    def load(self):
47
        # Load data from your source
48
        devices_data = fetch_devices_from_api()
49
        for device_data in devices_data:
50
            device = Device(**device_data)
51
            self.add(device)
52

53
# Create adapter instances
54
source_net = NetworkAdapter(name="source_network")
55
source_net.load()
56
```
57

58
### Data Loading
59

60
Methods for populating adapters with data from various sources.
61

62
```python { .api }
63
def load(self) -> None:
64
    """Load all desired data from whatever backend data source into this instance."""
65
```
66

67
```python { .api }
68
def load_from_dict(self, data: Dict) -> None:
69
    """
70
    The reverse of dict method, taking a dictionary and loading into the inventory.
71
    
72
    Args:
73
        data: Dictionary in the format that dict would export as
74
    """
75
```
76

77
#### Loading Example
78

79
```python
80
class DatabaseAdapter(Adapter):
81
    device = Device
82
    interface = Interface
83
    top_level = ["device"]
84
    
85
    def load(self):
86
        # Load from database
87
        with get_db_connection() as conn:
88
            # Load devices
89
            devices = conn.execute("SELECT * FROM devices").fetchall()
90
            for device_row in devices:
91
                device = Device(
92
                    name=device_row['name'],
93
                    vendor=device_row['vendor'],
94
                    model=device_row['model']
95
                )
96
                self.add(device)
97
            
98
            # Load interfaces
99
            interfaces = conn.execute("SELECT * FROM interfaces").fetchall()
100
            for intf_row in interfaces:
101
                interface = Interface(
102
                    device_name=intf_row['device_name'],
103
                    name=intf_row['name'],
104
                    ip_address=intf_row['ip_address']
105
                )
106
                # Add interface to its parent device
107
                device = self.get("device", intf_row['device_name'])
108
                device.add_child(interface)
109
                self.add(interface)
110
```
111

112
### Object Storage Operations
113

114
Methods for adding, retrieving, updating, and removing model instances from the adapter.
115

116
```python { .api }
117
def add(self, obj: DiffSyncModel) -> None:
118
    """
119
    Add a DiffSyncModel object to the store.
120
    
121
    Args:
122
        obj: Object to store
123
    
124
    Raises:
125
        ObjectAlreadyExists: if a different object with the same uid is already present
126
    """
127
```
128

129
```python { .api }
130
def update(self, obj: DiffSyncModel) -> None:
131
    """
132
    Update a DiffSyncModel object to the store.
133
    
134
    Args:
135
        obj: Object to store
136
    
137
    Raises:
138
        ObjectAlreadyExists: if a different object with the same uid is already present
139
    """
140
```
141

142
```python { .api }
143
def remove(self, obj: DiffSyncModel, remove_children: bool = False) -> None:
144
    """
145
    Remove a DiffSyncModel object from the store.
146
    
147
    Args:
148
        obj: object to remove
149
        remove_children: If True, also recursively remove any children of this object
150
    
151
    Raises:
152
        ObjectNotFound: if the object is not present
153
    """
154
```
155

156
### Object Retrieval
157

158
Methods for querying and retrieving stored model instances.
159

160
```python { .api }
161
def get(self, obj: Union[str, DiffSyncModel, Type[DiffSyncModel]],
162
        identifier: Union[str, Dict]) -> DiffSyncModel:
163
    """
164
    Get one object from the data store based on its unique id.
165
    
166
    Args:
167
        obj: DiffSyncModel class or instance, or modelname string, that defines the type of the object to retrieve
168
        identifier: Unique ID of the object to retrieve, or dict of unique identifier keys/values
169
    
170
    Raises:
171
        ValueError: if obj is a str and identifier is a dict (can't convert dict into a uid str without a model class)
172
        ObjectNotFound: if the requested object is not present
173
    """
174
```
175

176
```python { .api }
177
def get_or_none(self, obj: Union[str, DiffSyncModel, Type[DiffSyncModel]],
178
                identifier: Union[str, Dict]) -> Optional[DiffSyncModel]:
179
    """
180
    Get one object from the data store based on its unique id or get a None.
181
    
182
    Args:
183
        obj: DiffSyncModel class or instance, or modelname string, that defines the type of the object to retrieve
184
        identifier: Unique ID of the object to retrieve, or dict of unique identifier keys/values
185
    
186
    Returns:
187
        DiffSyncModel matching provided criteria or None if not found
188
    """
189
```
190

191
```python { .api }
192
def get_all(self, obj: Union[str, DiffSyncModel, Type[DiffSyncModel]]) -> List[DiffSyncModel]:
193
    """
194
    Get all objects of a given type.
195
    
196
    Args:
197
        obj: DiffSyncModel class or instance, or modelname string, that defines the type of the objects to retrieve
198
    
199
    Returns:
200
        List of Object
201
    """
202
```
203

204
```python { .api }
205
def get_by_uids(self, uids: List[str],
206
                obj: Union[str, DiffSyncModel, Type[DiffSyncModel]]) -> List[DiffSyncModel]:
207
    """
208
    Get multiple objects from the store by their unique IDs/Keys and type.
209
    
210
    Args:
211
        uids: List of unique id / key identifying object in the database
212
        obj: DiffSyncModel class or instance, or modelname string, that defines the type of the objects to retrieve
213
    
214
    Raises:
215
        ObjectNotFound: if any of the requested UIDs are not found in the store
216
    """
217
```
218

219
#### Retrieval Examples
220

221
```python
222
# Get specific device by name
223
device = adapter.get("device", "router1")
224
# Or using dict identifier
225
device = adapter.get(Device, {"name": "router1"})
226

227
# Get all devices
228
all_devices = adapter.get_all("device")
229

230
# Get multiple devices by IDs
231
device_ids = ["router1", "switch1", "firewall1"]
232
devices = adapter.get_by_uids(device_ids, "device")
233

234
# Safe get that returns None if not found
235
device = adapter.get_or_none("device", "nonexistent")
236
```
237

238
### Advanced Retrieval Operations
239

240
Convenience methods for more complex retrieval and management patterns.
241

242
```python { .api }
243
def get_or_instantiate(self, model: Type[DiffSyncModel], ids: Dict,
244
                      attrs: Optional[Dict] = None) -> Tuple[DiffSyncModel, bool]:
245
    """
246
    Attempt to get the object with provided identifiers or instantiate and add it with provided identifiers and attrs.
247
    
248
    Args:
249
        model: The DiffSyncModel to get or create
250
        ids: Identifiers for the DiffSyncModel to get or create with
251
        attrs: Attributes when creating an object if it doesn't exist
252
    
253
    Returns:
254
        Tuple of (existing or new object, whether it was created)
255
    """
256
```
257

258
```python { .api }
259
def get_or_add_model_instance(self, obj: DiffSyncModel) -> Tuple[DiffSyncModel, bool]:
260
    """
261
    Attempt to get the object with provided obj identifiers or add obj.
262
    
263
    Args:
264
        obj: An obj of the DiffSyncModel to get or add
265
    
266
    Returns:
267
        Tuple of (existing or new object, whether it was created)
268
    """
269
```
270

271
```python { .api }
272
def update_or_instantiate(self, model: Type[DiffSyncModel], ids: Dict,
273
                         attrs: Dict) -> Tuple[DiffSyncModel, bool]:
274
    """
275
    Attempt to update an existing object with provided ids/attrs or instantiate it with provided identifiers and attrs.
276
    
277
    Args:
278
        model: The DiffSyncModel to update or create
279
        ids: Identifiers for the DiffSyncModel to update or create with
280
        attrs: Attributes when creating/updating an object if it doesn't exist
281
    
282
    Returns:
283
        Tuple of (existing or new object, whether it was created)
284
    """
285
```
286

287
```python { .api }
288
def update_or_add_model_instance(self, obj: DiffSyncModel) -> Tuple[DiffSyncModel, bool]:
289
    """
290
    Attempt to update an existing object with provided obj ids/attrs or instantiate obj.
291
    
292
    Args:
293
        obj: An instance of the DiffSyncModel to update or create
294
    
295
    Returns:
296
        Tuple of (existing or new object, whether it was created)
297
    """
298
```
299

300
### Information and Statistics
301

302
Methods for getting information about the adapter's contents and structure.
303

304
```python { .api }
305
def get_all_model_names(self) -> Set[str]:
306
    """
307
    Get all model names.
308
    
309
    Returns:
310
        List of model names
311
    """
312
```
313

314
```python { .api }
315
def count(self, model: Union[str, "DiffSyncModel", Type["DiffSyncModel"], None] = None) -> int:
316
    """
317
    Count how many objects of one model type exist in the backend store.
318
    
319
    Args:
320
        model: The DiffSyncModel to check the number of elements. If not provided, default to all
321
    
322
    Returns:
323
        Number of elements of the model type
324
    """
325
```
326

327
```python { .api }
328
@classmethod
329
def get_tree_traversal(cls, as_dict: bool = False) -> Union[str, Dict]:
330
    """
331
    Get a string describing the tree traversal for the diffsync object.
332
    
333
    Args:
334
        as_dict: Whether to return as a dictionary
335
    
336
    Returns:
337
        A string or dictionary representation of tree
338
    """
339
```
340

341
### Serialization
342

343
Methods for converting adapter contents to various serializable formats.
344

345
```python { .api }
346
def dict(self, exclude_defaults: bool = True, **kwargs: Any) -> Dict[str, Dict[str, Dict]]:
347
    """Represent the DiffSync contents as a dict, as if it were a Pydantic model."""
348
```
349

350
```python { .api }
351
def str(self, indent: int = 0) -> str:
352
    """Build a detailed string representation of this Adapter."""
353
```
354

355
#### Serialization Example
356

357
```python
358
# Convert adapter to dictionary
359
adapter_data = adapter.dict()
360

361
# Convert to string representation
362
adapter_string = adapter.str()
363

364
# Save and load adapter state
365
import json
366
with open('adapter_state.json', 'w') as f:
367
    json.dump(adapter_data, f)
368

369
# Load back into new adapter
370
with open('adapter_state.json', 'r') as f:
371
    data = json.load(f)
372

373
new_adapter = NetworkAdapter()
374
new_adapter.load_from_dict(data)
375
```
376

377
## Types
378

379
```python { .api }
380
from typing import Any, Dict, List, Optional, Set, Tuple, Type, Union, ClassVar
381
from diffsync.store import BaseStore, LocalStore
382

383
# Storage engine type for adapter initialization
384
StorageEngine = Union[Type[BaseStore], BaseStore]
385
```