0
# Pool Management
1
2
Create and manage execution pools with different configurations for various types of workloads. Pools control how many tasks can run concurrently and which execution engine to use.
3
4
## Capabilities
5
6
### Pool Creation
7
8
Create new execution pools with custom thread counts and engine types.
9
10
```python { .api }
11
def createPool(name: str = "main", threads: Optional[int] = None, engine: Optional[str] = None) -> None:
12
"""
13
Create a new execution pool with specified configuration.
14
15
Pools manage concurrent task execution using semaphores. Each pool
16
has its own thread/process limit and engine type. Creating a pool
17
automatically makes it the active pool for new tasks.
18
19
Args:
20
name: Unique identifier for this pool
21
threads: Max concurrent tasks. None uses global MAX_THREADS.
22
Values < 2 create unlimited pools (no semaphore).
23
engine: "process" or "thread". None uses global ENGINE setting.
24
25
Note:
26
Setting threads=0 or threads=1 creates an unlimited pool where
27
all tasks run immediately without queuing.
28
"""
29
```
30
31
**Usage Example:**
32
33
```python
34
import multitasking
35
36
# Create different pools for different workloads
37
multitasking.createPool("api_calls", threads=20, engine="thread")
38
multitasking.createPool("cpu_intensive", threads=4, engine="process")
39
multitasking.createPool("unlimited", threads=0, engine="thread")
40
41
# The most recently created pool becomes active
42
@multitasking.task
43
def process_data(data):
44
# This will use the "unlimited" pool
45
pass
46
47
# Tasks will use whatever pool was active when they were defined
48
```
49
50
### Pool Information Retrieval
51
52
Get information about existing execution pools.
53
54
```python { .api }
55
def getPool(name: Optional[str] = None) -> Dict[str, Union[str, int]]:
56
"""
57
Retrieve information about an execution pool.
58
59
Returns a dictionary with pool metadata including engine type,
60
name, and thread count. Useful for debugging and monitoring.
61
62
Args:
63
name: Pool name to query. If None, uses current active pool.
64
65
Returns:
66
Dictionary with keys: 'engine', 'name', 'threads'
67
68
Raises:
69
KeyError: If the specified pool doesn't exist
70
"""
71
```
72
73
**Usage Example:**
74
75
```python
76
import multitasking
77
78
# Create some pools
79
multitasking.createPool("api_pool", threads=10, engine="thread")
80
multitasking.createPool("cpu_pool", threads=2, engine="process")
81
82
# Get information about specific pools
83
api_info = multitasking.getPool("api_pool")
84
print(f"API Pool: {api_info}")
85
# Output: {'engine': 'thread', 'name': 'api_pool', 'threads': 10}
86
87
cpu_info = multitasking.getPool("cpu_pool")
88
print(f"CPU Pool: {cpu_info}")
89
# Output: {'engine': 'process', 'name': 'cpu_pool', 'threads': 2}
90
91
# Get current active pool info
92
current_info = multitasking.getPool()
93
print(f"Current Pool: {current_info}")
94
```
95
96
## Pool Usage Patterns
97
98
### Specialized Workloads
99
100
Create different pools for different types of operations:
101
102
```python
103
import multitasking
104
import requests
105
import time
106
107
# Fast pool for quick API calls
108
multitasking.createPool("fast_api", threads=50, engine="thread")
109
110
@multitasking.task
111
def fetch_url(url):
112
response = requests.get(url, timeout=5)
113
return response.status_code
114
115
# Switch to CPU pool for intensive work
116
multitasking.createPool("compute", threads=2, engine="process")
117
118
@multitasking.task
119
def heavy_computation(data):
120
# CPU-intensive processing
121
return sum(x*x for x in data)
122
123
# Use the appropriate pool for each task type
124
urls = ["http://example.com" for _ in range(10)]
125
for url in urls:
126
fetch_url(url) # Uses compute pool (last created)
127
128
# Switch back to api pool by recreating it
129
multitasking.createPool("fast_api", threads=50, engine="thread")
130
```
131
132
### Dynamic Pool Configuration
133
134
Adjust pool settings based on runtime conditions:
135
136
```python
137
import multitasking
138
139
# Start with conservative settings
140
multitasking.createPool("dynamic", threads=2, engine="thread")
141
142
def adjust_pool_size(load_factor):
143
if load_factor > 0.8:
144
# High load: increase parallelism
145
multitasking.createPool("dynamic", threads=20, engine="thread")
146
elif load_factor < 0.3:
147
# Low load: reduce resource usage
148
multitasking.createPool("dynamic", threads=5, engine="thread")
149
150
# Monitor and adjust
151
current_load = 0.9 # Example load metric
152
adjust_pool_size(current_load)
153
154
pool_info = multitasking.getPool("dynamic")
155
print(f"Adjusted to: {pool_info['threads']} threads")
156
```