0
# dill
1

2
dill is an extended Python serialization library that extends Python's pickle module to handle a broader range of Python objects including functions with yields, nested functions, lambdas, classes, and more exotic Python types. It provides a drop-in replacement for pickle with enhanced capabilities for serializing complex Python objects that standard pickle cannot handle, making it particularly useful for distributed computing, parallel processing, and saving interpreter sessions.
3

4
The library offers additional features like source code inspection, interactive pickling diagnostics, and the ability to save and restore complete interpreter sessions, with applications in scientific computing, debugging, and development tools that require comprehensive Python object serialization.
5

6
## Package Information
7

8
- **Package Name**: dill
9
- **Language**: Python  
10
- **Installation**: `pip install dill`
11
- **Optional Features**: `pip install dill[graph]` for object graph diagnostics, `pip install dill[profile]` for profiling tools
12

13
## Core Imports
14

15
```python
16
import dill
17
```
18

19
Common usage patterns:
20

21
```python
22
import dill as pickle  # Drop-in replacement for pickle
23
from dill import dump, dumps, load, loads  # Direct function imports
24
```
25

26
## Basic Usage
27

28
```python
29
import dill
30

31
# Basic serialization - works like pickle but handles more types
32
def example_function():
33
    return "Hello from dill!"
34

35
# Serialize function to bytes
36
serialized = dill.dumps(example_function)
37

38
# Deserialize function
39
restored_function = dill.loads(serialized)
40
result = restored_function()  # "Hello from dill!"
41

42
# Serialize to file
43
with open('function.pkl', 'wb') as f:
44
    dill.dump(example_function, f)
45

46
# Load from file
47
with open('function.pkl', 'rb') as f:
48
    loaded_function = dill.load(f)
49
```
50

51
## Architecture
52

53
dill is designed as an extended version of Python's pickle module, maintaining full compatibility while adding support for previously unpickleable objects. The architecture consists of:
54

55
- **Core Serialization Engine**: Extended pickler/unpickler classes that handle complex Python objects
56
- **Type Registry System**: Automatic detection and registration of new types for serialization
57
- **Session Management**: Capability to save and restore entire interpreter sessions including modules
58
- **Source Code Analysis**: Tools for inspecting and extracting source code from objects
59
- **Diagnostic Framework**: Utilities for identifying and debugging serialization issues
60

61
dill integrates seamlessly with the broader Python ecosystem, supporting distributed computing frameworks like multiprocessing, concurrent.futures, and third-party libraries like Celery and Dask.
62

63
## Capabilities
64

65
### Core Serialization
66

67
Primary serialization and deserialization functions that extend pickle's capabilities to handle complex Python objects including functions, classes, and nested structures.
68

69
```python { .api }
70
def dump(obj, file, protocol=None, byref=None, fmode=None, recurse=None, **kwds):
71
    """
72
    Serialize object to a file.
73
    
74
    Parameters:
75
    - obj: object to serialize
76
    - file: file-like object to write to
77
    - protocol: int, pickle protocol version (default: DEFAULT_PROTOCOL)
78
    - byref: bool, pickle by reference when possible
79
    - fmode: int, file mode for handle management
80
    - recurse: bool, recursively pickle nested objects
81
    - **kwds: additional keyword arguments
82
    
83
    Returns:
84
    None
85
    """
86

87
def dumps(obj, protocol=None, byref=None, fmode=None, recurse=None, **kwds):
88
    """
89
    Serialize object to a bytes string.
90
    
91
    Parameters:
92
    - obj: object to serialize
93
    - protocol: int, pickle protocol version (default: DEFAULT_PROTOCOL) 
94
    - byref: bool, pickle by reference when possible
95
    - fmode: int, file mode for handle management
96
    - recurse: bool, recursively pickle nested objects
97
    - **kwds: additional keyword arguments
98
    
99
    Returns:
100
    bytes: serialized object as bytes string
101
    """
102

103
def load(file, ignore=None, **kwds):
104
    """
105
    Deserialize object from a file.
106
    
107
    Parameters:
108
    - file: file-like object to read from
109
    - ignore: bool, ignore certain unpickling errors
110
    - **kwds: additional keyword arguments
111
    
112
    Returns:
113
    object: deserialized object
114
    """
115

116
def loads(str, ignore=None, **kwds):
117
    """
118
    Deserialize object from a bytes string.
119
    
120
    Parameters:
121
    - str: bytes string containing serialized object
122
    - ignore: bool, ignore certain unpickling errors  
123
    - **kwds: additional keyword arguments
124
    
125
    Returns:
126
    object: deserialized object
127
    """
128

129
def copy(obj, *args, **kwds):
130
    """
131
    Create a deep copy of an object using serialization.
132
    
133
    Parameters:
134
    - obj: object to copy
135
    - *args: positional arguments passed to dumps/loads
136
    - **kwds: keyword arguments passed to dumps/loads
137
    
138
    Returns:
139
    object: deep copy of the input object
140
    """
141
```
142

143
[Core Serialization](./core-serialization.md)
144

145
### Extended Pickler Classes
146

147
Enhanced Pickler and Unpickler classes that provide fine-grained control over the serialization process and support for complex Python objects.
148

149
```python { .api }
150
class Pickler:
151
    """
152
    Extended pickler with additional capabilities for complex objects.
153
    
154
    Provides enhanced serialization support beyond standard pickle.Pickler,
155
    including functions, classes, and other previously unpickleable types.
156
    """
157

158
class Unpickler:
159
    """
160
    Extended unpickler with additional capabilities for complex objects.
161
    
162
    Provides enhanced deserialization support beyond standard pickle.Unpickler,
163
    with improved error handling and type restoration.
164
    """
165
```
166

167
[Extended Pickler Classes](./pickler-classes.md)
168

169
### Session Management
170

171
Functions for saving and restoring complete interpreter sessions and individual modules, enabling persistence of development environments and interactive workflows.
172

173
```python { .api }
174
def dump_session(filename=None, main=None, byref=False, **kwds):
175
    """
176
    Save interpreter session to a file.
177
    
178
    Parameters:
179
    - filename: str, output filename (default: temporary file)
180
    - main: module, main module to save (default: __main__)
181
    - byref: bool, pickle by reference when possible
182
    - **kwds: additional keyword arguments
183
    
184
    Returns:
185
    None
186
    """
187

188
def load_session(filename=None, main=None, **kwds):
189
    """
190
    Load interpreter session from a file.
191
    
192
    Parameters:
193
    - filename: str, input filename
194
    - main: module, target module (default: __main__)
195
    - **kwds: additional keyword arguments
196
    
197
    Returns:
198
    None
199
    """
200

201
def dump_module(name=None, module=None, main=None, **kwds):
202
    """
203
    Save a module to a file.
204
    
205
    Parameters:
206
    - name: str, output filename
207
    - module: module, module object to save
208
    - main: module, main module context
209
    - **kwds: additional keyword arguments
210
    
211
    Returns:
212
    None
213
    """
214

215
def load_module(name=None, module=None, main=None, **kwds):
216
    """
217
    Load a module from a file.
218
    
219
    Parameters:
220
    - name: str, input filename
221
    - module: str, module name to load into
222
    - main: module, main module context
223
    - **kwds: additional keyword arguments
224
    
225
    Returns:
226
    module: loaded module object
227
    """
228
```
229

230
[Session Management](./session-management.md)
231

232
### Source Code Analysis
233

234
Tools for extracting, analyzing, and manipulating source code from Python objects, enabling introspection and code generation capabilities.
235

236
```python { .api }
237
def getsource(object, alias='', lstrip=False, enclosing=False, force=False, builtin=False):
238
    """
239
    Get source code for an object.
240
    
241
    Parameters:
242
    - object: object to get source for (module, class, method, function, traceback, frame, or code object)
243
    - alias: str, alias name for the object (adds line of code that renames the object)
244
    - lstrip: bool, ensure there is no indentation in the first line of code
245
    - enclosing: bool, include enclosing code and dependencies
246
    - force: bool, catch (TypeError,IOError) and try to use import hooks
247
    - builtin: bool, force an import for any builtins
248
    
249
    Returns:
250
    str: source code as single string
251
    
252
    Raises:
253
    - IOError: when source code cannot be retrieved
254
    - TypeError: for objects where source code is unavailable (e.g. builtins)
255
    """
256

257
def getimport(obj, alias='', verify=True, builtin=False, enclosing=False):
258
    """
259
    Get the likely import string for the given object.
260
    
261
    Parameters:
262
    - obj: object to inspect and generate import for
263
    - alias: str, alias name to use (renames the object on import)
264
    - verify: bool, test the import string before returning it
265
    - builtin: bool, force an import for builtins where possible
266
    - enclosing: bool, get the import for the outermost enclosing callable
267
    
268
    Returns:
269
    str: import statement as string
270
    """
271
```
272

273
[Source Code Analysis](./source-analysis.md)
274

275
### Diagnostic Tools
276

277
Utilities for analyzing serialization capabilities, identifying problems, and debugging pickling issues with detailed error reporting.
278

279
```python { .api }
280
def pickles(obj, exact=False, safe=False, **kwds):
281
    """
282
    Check if an object can be pickled.
283
    
284
    Parameters:
285
    - obj: object to test for pickling capability
286
    - exact: bool, use exact type matching for compatibility testing
287
    - safe: bool, use safe mode to avoid side effects during testing
288
    - **kwds: additional keyword arguments passed to dumps/loads
289
    
290
    Returns:
291
    bool: True if object can be pickled and unpickled successfully
292
    """
293

294
def check(obj, *args, **kwds):
295
    """
296
    Check for pickling errors and print diagnostic information.
297
    
298
    Parameters:
299
    - obj: object to check for pickling errors
300
    - *args: positional arguments passed to pickles()
301
    - **kwds: keyword arguments passed to pickles()
302
    
303
    Returns:
304
    bool: True if no errors found, False if pickling issues detected
305
    """
306

307
def baditems(obj, exact=False, safe=False):
308
    """
309
    Find objects that cannot be pickled within a complex structure.
310
    
311
    Parameters:
312
    - obj: object to analyze for unpickleable items
313
    - exact: bool, use exact type matching for analysis
314
    - safe: bool, use safe mode to avoid side effects
315
    
316
    Returns:
317
    list: list of unpickleable objects found in the structure
318
    """
319

320
def badobjects(obj, depth=0, exact=False, safe=False):
321
    """
322
    Get objects that fail to pickle.
323
    
324
    Parameters:
325
    - obj: object to analyze
326
    - depth: int, analysis depth (0 for immediate object only, >0 for recursive analysis)
327
    - exact: bool, use exact type matching
328
    - safe: bool, use safe mode to avoid side effects
329
    
330
    Returns:
331
    object or dict: at depth=0 returns the object if it fails to pickle (None if it pickles), 
332
    at depth>0 returns dict mapping attribute names to bad objects
333
    """
334

335
def errors(obj, depth=0, exact=False, safe=False):
336
    """
337
    Get detailed pickling error information.
338
    
339
    Parameters:
340
    - obj: object to analyze for errors
341
    - depth: int, analysis depth (0 for immediate object only, >0 for recursive analysis)
342
    - exact: bool, use exact type matching
343
    - safe: bool, use safe mode to avoid side effects
344
    
345
    Returns:
346
    Exception or dict: detailed error information for pickling failures
347
    """
348
```
349

350
[Diagnostic Tools](./diagnostic-tools.md)
351

352
### Type Registry System
353

354
Functions for registering custom types and extending dill's serialization capabilities to handle new object types.
355

356
```python { .api }
357
def register(t):
358
    """
359
    Register a type with the pickler.
360
    
361
    Parameters:
362
    - t: type, type to register
363
    
364
    Returns:
365
    function: decorator function
366
    """
367

368
def pickle(t, func):
369
    """
370
    Add a type to the pickle dispatch table.
371
    
372
    Parameters:
373
    - t: type, type to add
374
    - func: function, pickling function for the type
375
    
376
    Returns:
377
    None
378
    """
379

380
def extend(use_dill=True):
381
    """
382
    Add or remove dill types to/from the pickle registry.
383
    
384
    Parameters:
385
    - use_dill: bool, if True extend dispatch table, if False revert
386
    
387
    Returns:
388
    None
389
    """
390
```
391

392
[Type Registry System](./type-registry.md)
393

394
### Temporary Operations
395

396
Utilities for temporary file operations with serialization, stream capture, and IO buffer management for testing and development workflows.
397

398
```python { .api }
399
def dump(object, **kwds):
400
    """
401
    Dump object to a NamedTemporaryFile using dill.dump.
402
    
403
    Parameters:
404
    - object: object to serialize to temporary file
405
    - **kwds: optional keyword arguments including suffix, prefix, and NamedTemporaryFile options
406
    
407
    Returns:
408
    file handle: NamedTemporaryFile handle containing serialized object
409
    """
410

411
def load(file, **kwds):
412
    """
413
    Load an object that was stored with dill.temp.dump.
414
    
415
    Parameters:
416
    - file: file handle or str, file handle or path to file containing serialized object
417
    - **kwds: optional keyword arguments including mode ('r' or 'rb', default: 'rb')
418
    
419
    Returns:
420
    object: deserialized object
421
    """
422

423
def capture(stream='stdout'):
424
    """
425
    Capture stdout or stderr stream.
426
    
427
    Parameters:
428
    - stream: str, stream name ('stdout' or 'stderr')
429
    
430
    Returns:
431
    context manager for stream capture
432
    """
433
```
434

435
[Temporary Operations](./temp-operations.md)
436

437
### Configuration and Settings
438

439
Global configuration options and settings that control dill's behavior, protocol selection, and serialization modes.
440

441
```python { .api }
442
# Global settings dictionary
443
settings = {
444
    'protocol': DEFAULT_PROTOCOL,  # Default pickle protocol  
445
    'byref': False,               # Pickle by reference
446
    'fmode': 0,                   # File mode setting
447
    'recurse': False,             # Recursive pickling
448
    'ignore': False               # Ignore errors
449
}
450
```
451

452
[Configuration and Settings](./configuration.md)