0
# Pool Management
1
2
Sophisticated connection pool management for high-performance HTTP clients. Pool managers automatically handle connection pooling across different hosts and protocols, providing optimal performance for applications making many HTTP requests.
3
4
## Capabilities
5
6
### PoolManager
7
8
The main interface for managing connection pools across multiple hosts. Automatically creates and manages HTTP/HTTPS connection pools based on the target URL, providing connection reuse and thread-safe operation.
9
10
```python { .api }
11
class PoolManager:
12
def __init__(self, num_pools=10, headers=None, **connection_pool_kw):
13
"""
14
HTTP connection pool manager.
15
16
Parameters:
17
- num_pools: Number of connection pools to cache (default: 10)
18
- headers: Headers to add to all requests
19
- **connection_pool_kw: Additional keyword arguments for connection pools
20
"""
21
22
def request(self, method: str, url: str, **kw) -> BaseHTTPResponse:
23
"""
24
Make a request using appropriate connection pool.
25
26
Parameters:
27
- method: HTTP method (GET, POST, etc.)
28
- url: Complete URL including scheme and host
29
- **kw: Additional request arguments
30
31
Returns:
32
BaseHTTPResponse: Response object
33
"""
34
35
def urlopen(self, method: str, url: str, **kw) -> BaseHTTPResponse:
36
"""
37
Same as request() method - provided for compatibility.
38
"""
39
40
def clear(self):
41
"""
42
Empty the pooled connections and clear the cache.
43
"""
44
```
45
46
### ProxyManager
47
48
Specialized pool manager for making requests through HTTP or HTTPS proxies. Handles proxy authentication and tunnel setup automatically.
49
50
```python { .api }
51
class ProxyManager(PoolManager):
52
def __init__(self, proxy_url, num_pools=10, headers=None, **connection_pool_kw):
53
"""
54
HTTP proxy connection pool manager.
55
56
Parameters:
57
- proxy_url: URL of the proxy server (http://proxy.example.com:8080)
58
- num_pools: Number of connection pools to cache
59
- headers: Headers to add to all requests
60
- **connection_pool_kw: Additional keyword arguments for connection pools
61
"""
62
```
63
64
### Proxy Creation Utility
65
66
```python { .api }
67
def proxy_from_url(url: str, **kw) -> ProxyManager:
68
"""
69
Create a ProxyManager from a proxy URL.
70
71
Parameters:
72
- url: Proxy URL (e.g., 'http://proxy.example.com:8080')
73
- **kw: Additional keyword arguments for ProxyManager
74
75
Returns:
76
ProxyManager: Configured proxy manager
77
"""
78
```
79
80
## Usage Examples
81
82
### Basic PoolManager Usage
83
84
```python
85
import urllib3
86
87
# Create a pool manager
88
http = urllib3.PoolManager()
89
90
# Make requests - pools are created automatically
91
resp1 = http.request('GET', 'https://httpbin.org/get')
92
resp2 = http.request('GET', 'https://jsonplaceholder.typicode.com/posts/1')
93
resp3 = http.request('POST', 'https://httpbin.org/post', fields={'key': 'value'})
94
95
print(f"Response 1: {resp1.status}")
96
print(f"Response 2: {resp2.status}")
97
print(f"Response 3: {resp3.status}")
98
```
99
100
### Custom Headers and Configuration
101
102
```python
103
import urllib3
104
105
# Pool manager with default headers
106
http = urllib3.PoolManager(
107
headers={'User-Agent': 'MyApp/1.0'},
108
timeout=urllib3.Timeout(connect=1.0, read=2.0),
109
retries=urllib3.Retry(total=3)
110
)
111
112
resp = http.request('GET', 'https://httpbin.org/headers')
113
```
114
115
### HTTPS with Custom SSL Context
116
117
```python
118
import urllib3
119
import ssl
120
121
# Create custom SSL context
122
ctx = ssl.create_default_context()
123
ctx.check_hostname = False
124
ctx.verify_mode = ssl.CERT_NONE
125
126
# Pool manager with custom SSL context
127
http = urllib3.PoolManager(ssl_context=ctx)
128
resp = http.request('GET', 'https://self-signed.badssl.com/')
129
```
130
131
### Proxy Usage
132
133
```python
134
import urllib3
135
136
# Create proxy manager
137
proxy = urllib3.ProxyManager('http://proxy.example.com:8080')
138
139
# All requests go through the proxy
140
resp = proxy.request('GET', 'https://httpbin.org/get')
141
142
# Or use proxy_from_url utility
143
proxy = urllib3.proxy_from_url('http://proxy.example.com:8080')
144
resp = proxy.request('GET', 'https://httpbin.org/get')
145
```
146
147
### Proxy with Authentication
148
149
```python
150
import urllib3
151
152
# Proxy with authentication
153
proxy = urllib3.ProxyManager(
154
'http://username:password@proxy.example.com:8080'
155
)
156
157
resp = proxy.request('GET', 'https://httpbin.org/get')
158
```
159
160
### Connection Pool Configuration
161
162
```python
163
import urllib3
164
165
# Pool manager with custom connection pool settings
166
http = urllib3.PoolManager(
167
num_pools=20, # Cache up to 20 different host pools
168
maxsize=10, # Max 10 connections per pool
169
block=True, # Block when pool is full
170
timeout=urllib3.Timeout(connect=1.0, read=5.0),
171
retries=urllib3.Retry(total=3, backoff_factor=0.3)
172
)
173
174
# Make many requests efficiently
175
for i in range(100):
176
resp = http.request('GET', f'https://httpbin.org/get?id={i}')
177
print(f"Request {i}: {resp.status}")
178
```
179
180
### Managing Pool Lifecycle
181
182
```python
183
import urllib3
184
185
http = urllib3.PoolManager()
186
187
# Use the pool manager
188
resp = http.request('GET', 'https://httpbin.org/get')
189
190
# Clear all pooled connections when done
191
http.clear()
192
```
193
194
## Performance Benefits
195
196
- **Connection Reuse**: Automatically reuses connections to the same host
197
- **Thread Safety**: Safe to use from multiple threads
198
- **Automatic Pool Management**: Creates and destroys pools based on usage
199
- **Protocol Support**: Handles both HTTP and HTTPS automatically
200
- **Keep-Alive**: Maintains persistent connections when supported by server