0
# Multiprocess
1

2
A fork of Python's multiprocessing module that extends multiprocessing to provide enhanced serialization using dill. Multiprocess leverages multiprocessing to support the spawning of processes using the API of the Python standard library's threading module, with better object serialization capabilities for complex Python objects.
3

4
## Package Information
5

6
- **Package Name**: multiprocess
7
- **Language**: Python
8
- **Installation**: `pip install multiprocess`
9
- **Python Support**: 3.9-3.14, PyPy 3.9-3.11
10
- **Dependencies**: dill>=0.4.0, setuptools>=42
11

12
## Core Imports
13

14
```python
15
import multiprocess
16
```
17

18
Common imports for specific functionality:
19

20
```python
21
from multiprocess import Process, Queue, Pool
22
from multiprocess import Lock, Semaphore, Event, Condition
23
from multiprocess import Manager, Pipe
24
```
25

26
For shared memory functionality:
27

28
```python
29
from multiprocess.shared_memory import SharedMemory, ShareableList
30
from multiprocess.sharedctypes import copy, synchronized
31
```
32

33
## Basic Usage
34

35
```python
36
from multiprocess import Process, Queue
37

38
def worker(q):
39
    q.put('hello world')
40

41
if __name__ == '__main__':
42
    q = Queue()
43
    p = Process(target=worker, args=[q])
44
    p.start()
45
    print(q.get())  # Output: hello world
46
    p.join()
47
```
48

49
## Architecture
50

51
Multiprocess maintains full API compatibility with Python's built-in multiprocessing module while adding enhanced serialization capabilities:
52

53
- **Process Management**: Create and control worker processes using Process class
54
- **Synchronization**: Thread-like synchronization primitives (Lock, Semaphore, Event, etc.)
55
- **Communication**: Inter-process communication via Queues, Pipes, and shared objects
56
- **Pool Processing**: Parallel task execution using worker process pools
57
- **Enhanced Serialization**: Extended object serialization using dill for complex objects
58
- **Context Management**: Support for different process start methods (fork, spawn, forkserver)
59

60
The enhanced serialization using dill allows multiprocess to handle complex Python objects that standard multiprocessing cannot serialize, including lambda functions, nested functions, and complex class instances.
61

62
## Capabilities
63

64
### Process Management
65

66
Core functionality for creating, managing, and controlling processes. Provides the Process class with threading-like API, along with utilities for process inspection and lifecycle management.
67

68
```python { .api }
69
class Process:
70
    def __init__(self, target=None, name=None, args=(), kwargs={}, daemon=None): ...
71
    def start(self): ...
72
    def join(self, timeout=None): ...
73
    def terminate(self): ...
74
    def kill(self): ...
75
    def is_alive(self): ...
76

77
def current_process(): ...
78
def active_children(): ...
79
def parent_process(): ...
80
def cpu_count(): ...
81
```
82

83
[Process Management](./process-management.md)
84

85
### Synchronization Primitives
86

87
Thread-like synchronization objects for coordinating processes. Includes locks, semaphores, events, conditions, and barriers for process synchronization and mutual exclusion.
88

89
```python { .api }
90
def Lock(): ...
91
def RLock(): ...
92
def Semaphore(value=1): ...
93
def BoundedSemaphore(value=1): ...
94
def Event(): ...
95
def Condition(lock=None): ...
96
def Barrier(parties, action=None, timeout=None): ...
97
```
98

99
[Synchronization Primitives](./synchronization.md)
100

101
### Inter-Process Communication
102

103
Communication mechanisms for data exchange between processes. Provides queues for message passing and pipes for bidirectional communication with various queue types and connection objects.
104

105
```python { .api }
106
class Queue:
107
    def __init__(self, maxsize=0): ...
108
    def put(self, item, block=True, timeout=None): ...
109
    def get(self, block=True, timeout=None): ...
110
    def empty(self): ...
111
    def full(self): ...
112

113
class JoinableQueue(Queue):
114
    def task_done(self): ...
115
    def join(self): ...
116

117
class SimpleQueue:
118
    def put(self, item): ...
119
    def get(self): ...
120
    def empty(self): ...
121

122
def Pipe(duplex=True): ...
123
```
124

125
[Inter-Process Communication](./communication.md)
126

127
### Process Pools
128

129
Parallel task execution using worker process pools. Provides Pool class for distributing tasks across multiple processes with various execution patterns and async result handling.
130

131
```python { .api }
132
class Pool:
133
    def __init__(self, processes=None, initializer=None, initargs=(), 
134
                 maxtasksperchild=None): ...
135
    def map(self, func, iterable, chunksize=None): ...
136
    def map_async(self, func, iterable, chunksize=None, callback=None, 
137
                  error_callback=None): ...
138
    def imap(self, func, iterable, chunksize=1): ...
139
    def imap_unordered(self, func, iterable, chunksize=1): ...
140
    def starmap(self, func, iterable, chunksize=None): ...
141
    def starmap_async(self, func, iterable, chunksize=None, callback=None, 
142
                      error_callback=None): ...
143
    def apply(self, func, args=(), kwds={}): ...
144
    def apply_async(self, func, args=(), kwds={}, callback=None, 
145
                    error_callback=None): ...
146
    def close(self): ...
147
    def terminate(self): ...
148
    def join(self): ...
149
```
150

151
[Process Pools](./pools.md)
152

153
### Shared Objects and Memory
154

155
Objects and memory that can be shared between processes. Includes both high-level managed objects and low-level shared memory constructs for different sharing patterns and performance requirements.
156

157
```python { .api }
158
def Manager(): ...
159

160
class SharedMemory:
161
    def __init__(self, name=None, create=False, size=0): ...
162
    def close(self): ...
163
    def unlink(self): ...
164

165
class ShareableList:
166
    def __init__(self, sequence=None, name=None): ...
167
    def copy(self): ...
168
    def count(self, value): ...
169
    def index(self, value): ...
170

171
def Value(typecode_or_type, *args, lock=True): ...
172
def Array(typecode_or_type, size_or_initializer, lock=True): ...
173
def RawValue(typecode_or_type, *args): ...
174
def RawArray(typecode_or_type, size_or_initializer): ...
175
```
176

177
[Shared Objects and Memory](./shared-objects.md)
178

179
### Context and Configuration
180

181
Context management and configuration options for different process start methods and serialization behavior. Includes logging utilities, debugging support, and advanced configuration for process creation and object serialization.
182

183
```python { .api }
184
def get_context(method=None): ...
185
def set_start_method(method, force=False): ...
186
def get_start_method(allow_none=False): ...
187
def get_all_start_methods(): ...
188
def set_executable(executable): ...
189
def set_forkserver_preload(module_names): ...
190
def get_logger(): ...
191
def log_to_stderr(level=None): ...
192
def freeze_support(): ...
193
def allow_connection_pickling(): ...
194
```
195

196
[Context and Configuration](./context-config.md)
197

198
## Exception Types
199

200
```python { .api }
201
class ProcessError(Exception): ...
202
class BufferTooShort(ProcessError): ...
203
class TimeoutError(ProcessError): ...
204
class AuthenticationError(ProcessError): ...
205
```
206

207
## Constants
208

209
```python { .api }
210
SUBDEBUG: int = 5
211
SUBWARNING: int = 25
212
```