0
# Pebble
1

2
Threading and multiprocessing eye-candy providing a clean, decorator-based API for concurrent execution. Pebble offers thread and process pools with advanced features like timeouts, worker restart capabilities, and error handling with traceback preservation.
3

4
## Package Information
5

6
- **Package Name**: pebble
7
- **Language**: Python
8
- **Installation**: `pip install pebble`
9

10
## Core Imports
11

12
```python
13
import pebble
14
```
15

16
Common imports for decorators:
17

18
```python
19
from pebble.concurrent import thread, process
20
from pebble.asynchronous import thread, process
21
```
22

23
Common imports for pools:
24

25
```python
26
from pebble import ThreadPool, ProcessPool
27
```
28

29
Common imports for utilities:
30

31
```python
32
from pebble import synchronized, waitforthreads, waitforqueues
33
```
34

35
## Basic Usage
36

37
```python
38
from pebble.concurrent import process
39
from pebble import ProcessPool
40
import time
41

42
# Using decorators for simple concurrent execution
43
@process
44
def cpu_intensive_task(n):
45
    # Simulate CPU-intensive work
46
    total = 0
47
    for i in range(n):
48
        total += i ** 2
49
    return total
50

51
# Call returns a Future
52
future = cpu_intensive_task(1000000)
53
result = future.result()  # Blocks until complete
54
print(f"Result: {result}")
55

56
# Using pools for managing multiple workers
57
with ProcessPool() as pool:
58
    # Schedule multiple tasks
59
    futures = []
60
    for i in range(5):
61
        future = pool.schedule(cpu_intensive_task, args=[100000 * (i + 1)])
62
        futures.append(future)
63
    
64
    # Collect results
65
    results = [f.result() for f in futures]
66
    print(f"Results: {results}")
67
```
68

69
## Architecture
70

71
Pebble's architecture centers around two execution models:
72

73
- **Decorator-based execution**: Simple decorators that wrap functions for concurrent execution, returning Future objects
74
- **Pool-based execution**: Managed worker pools that provide fine-grained control over resource allocation and lifecycle
75

76
Key components:
77

78
- **Concurrent decorators**: Synchronous decorators returning `concurrent.futures.Future`
79
- **Asynchronous decorators**: AsyncIO-compatible decorators returning `asyncio.Future`
80
- **Pool classes**: Managed pools of workers with scheduling, timeout, and lifecycle management
81
- **Future types**: Enhanced Future objects with process-specific capabilities
82
- **Utility functions**: Helper functions for synchronization and waiting operations
83

84
## Capabilities
85

86
### Concurrent Decorators
87

88
Synchronous decorators for thread and process-based execution that return `concurrent.futures.Future` objects. These decorators provide the simplest way to execute functions concurrently with minimal code changes.
89

90
```python { .api }
91
@thread(name=None, daemon=True, pool=None)
92
def decorated_function(): ...
93

94
@process(name=None, daemon=True, timeout=None, mp_context=None, pool=None)
95
def decorated_function(): ...
96
```
97

98
[Concurrent Decorators](./concurrent-decorators.md)
99

100
### Asynchronous Decorators
101

102
AsyncIO-compatible decorators for thread and process-based execution that return `asyncio.Future` objects. Perfect for integration with async/await patterns and AsyncIO applications.
103

104
```python { .api }
105
@thread(name=None, daemon=True, pool=None)
106
async def decorated_function(): ...
107

108
@process(name=None, daemon=True, timeout=None, mp_context=None, pool=None)
109
async def decorated_function(): ...
110
```
111

112
[Asynchronous Decorators](./asynchronous-decorators.md)
113

114
### Thread Pools
115

116
Managed pools of worker threads for executing multiple tasks concurrently. Thread pools are ideal for I/O-bound tasks and provide fine-grained control over worker lifecycle and task scheduling.
117

118
```python { .api }
119
class ThreadPool:
120
    def __init__(self, max_workers=multiprocessing.cpu_count(), max_tasks=0, initializer=None, initargs=[]): ...
121
    def schedule(self, function, args=(), kwargs={}): ...
122
    def submit(self, function, *args, **kwargs): ...
123
    def map(self, function, *iterables, **kwargs): ...
124
    def close(self): ...
125
    def stop(self): ...
126
    def join(self): ...
127
```
128

129
[Thread Pools](./thread-pools.md)
130

131
### Process Pools
132

133
Managed pools of worker processes for executing CPU-intensive tasks. Process pools bypass Python's GIL and provide true parallelism with advanced features like timeouts and automatic worker restart.
134

135
```python { .api }
136
class ProcessPool:
137
    def __init__(self, max_workers=multiprocessing.cpu_count(), max_tasks=0, initializer=None, initargs=[], context=multiprocessing): ...
138
    def schedule(self, function, args=(), kwargs={}, timeout=None): ...
139
    def submit(self, function, timeout, /, *args, **kwargs): ...
140
    def map(self, function, *iterables, **kwargs): ...
141
    def close(self): ...
142
    def stop(self): ...
143
    def join(self): ...
144
```
145

146
[Process Pools](./process-pools.md)
147

148
### Synchronization Utilities
149

150
Utility functions and decorators for thread synchronization, signal handling, and waiting operations. These tools help coordinate concurrent execution and handle edge cases.
151

152
```python { .api }
153
@synchronized(lock=None)
154
def decorated_function(): ...
155

156
@sighandler(signals)
157
def signal_handler(): ...
158

159
def waitforthreads(threads, timeout=None): ...
160
def waitforqueues(queues, timeout=None): ...
161
```
162

163
[Synchronization Utilities](./synchronization-utilities.md)
164

165
### Future Types and Exceptions
166

167
Enhanced Future objects and exception types for handling concurrent execution results, timeouts, and error conditions specific to Pebble's execution model.
168

169
```python { .api }
170
class ProcessFuture(concurrent.futures.Future):
171
    def cancel(self): ...
172
    def result(self, timeout=None): ...
173
    def exception(self, timeout=None): ...
174

175
class ProcessExpired(OSError):
176
    def __init__(self, msg, code=0, pid=None): ...
177

178
class MapFuture(concurrent.futures.Future): ...
179
class ProcessMapFuture(concurrent.futures.Future): ...
180
```
181

182
[Future Types and Exceptions](./future-types-exceptions.md)
183

184
## Types
185

186
```python { .api }
187
# Core execution types
188
from typing import Callable, Optional, Any, List, Dict
189
from concurrent.futures import Future
190
from multiprocessing.context import BaseContext
191
from threading import Lock, RLock, Semaphore
192
import signal
193

194
# Function signatures for decorators
195
CallableType = Callable[..., Any]
196
ThreadDecoratorReturnType = Callable[..., Future]
197
ProcessDecoratorReturnType = Callable[..., 'ProcessFuture']
198
AsyncIODecoratorReturnType = Callable[..., 'asyncio.Future']
199

200
# Parameter types
201
ThreadDecoratorParams = {
202
    'name': Optional[str],
203
    'daemon': bool,
204
    'pool': Optional['ThreadPool']
205
}
206

207
ProcessDecoratorParams = {
208
    'name': Optional[str], 
209
    'daemon': bool,
210
    'timeout': Optional[float],
211
    'mp_context': Optional[BaseContext],
212
    'pool': Optional['ProcessPool']
213
}
214

215
# Synchronization types
216
SynchronizationLock = Lock | RLock | Semaphore
217
SignalType = int | List[int]
218
```