0
# Connection Pools
1
2
Direct connection pool management for specific hosts with fine-grained control over connection behavior, SSL/TLS settings, and per-host configuration. Connection pools provide the foundation for urllib3's performance and reliability.
3
4
## Capabilities
5
6
### HTTPConnectionPool
7
8
Manages a pool of HTTP connections to a specific host, providing connection reuse, thread safety, and configurable behavior for handling connection failures and retries.
9
10
```python { .api }
11
class HTTPConnectionPool:
12
def __init__(self, host: str, port=None, timeout=None, maxsize=1,
13
block=False, headers=None, retries=None, **conn_kw):
14
"""
15
HTTP connection pool for a specific host.
16
17
Parameters:
18
- host: Host to connect to
19
- port: Port number (default: 80 for HTTP, 443 for HTTPS)
20
- timeout: Default timeout for requests (float, Timeout object, or None)
21
- maxsize: Maximum number of connections in the pool
22
- block: Block when pool is full instead of raising FullPoolError
23
- headers: Default headers for all requests
24
- retries: Default retry configuration (Retry object or None)
25
- **conn_kw: Additional connection arguments
26
"""
27
28
def urlopen(self, method: str, url: str, **kw) -> BaseHTTPResponse:
29
"""
30
Make a request using a connection from the pool.
31
32
Parameters:
33
- method: HTTP method (GET, POST, etc.)
34
- url: Path portion of URL (not including host)
35
- **kw: Additional request arguments
36
37
Returns:
38
BaseHTTPResponse: Response object
39
"""
40
41
def close(self):
42
"""
43
Close all pooled connections and disable the pool.
44
"""
45
46
def is_same_host(self, url: str) -> bool:
47
"""
48
Check if URL is for the same host as this pool.
49
"""
50
```
51
52
### HTTPSConnectionPool
53
54
Extends HTTPConnectionPool with SSL/TLS support, certificate verification, and HTTPS-specific configuration options.
55
56
```python { .api }
57
class HTTPSConnectionPool(HTTPConnectionPool):
58
def __init__(self, host: str, port=None, timeout=None, maxsize=1,
59
block=False, headers=None, retries=None, **conn_kw):
60
"""
61
HTTPS connection pool with SSL/TLS support.
62
63
Additional SSL-specific parameters in conn_kw:
64
- ssl_context: Custom SSL context
65
- cert_file: Path to client certificate file
66
- key_file: Path to client private key file
67
- cert_reqs: Certificate requirements (ssl.CERT_NONE, ssl.CERT_REQUIRED)
68
- ca_certs: Path to CA certificate bundle
69
- ssl_version: SSL/TLS version to use
70
- assert_hostname: Hostname to verify in certificate
71
- assert_fingerprint: Expected certificate fingerprint
72
"""
73
```
74
75
### Connection Pool Creation Utility
76
77
```python { .api }
78
def connection_from_url(url: str, **kw) -> HTTPConnectionPool:
79
"""
80
Create an appropriate connection pool from a URL.
81
82
Parameters:
83
- url: Complete URL (scheme determines HTTP vs HTTPS pool)
84
- **kw: Additional arguments for connection pool
85
86
Returns:
87
HTTPConnectionPool or HTTPSConnectionPool: Configured connection pool
88
"""
89
```
90
91
## Usage Examples
92
93
### Basic HTTP Connection Pool
94
95
```python
96
import urllib3
97
98
# Create HTTP connection pool
99
pool = urllib3.HTTPConnectionPool('httpbin.org', port=80)
100
101
# Make requests using the pool
102
resp1 = pool.urlopen('GET', '/get')
103
resp2 = pool.urlopen('POST', '/post', fields={'key': 'value'})
104
105
print(f"GET response: {resp1.status}")
106
print(f"POST response: {resp2.status}")
107
108
# Close the pool when done
109
pool.close()
110
```
111
112
### HTTPS Connection Pool
113
114
```python
115
import urllib3
116
117
# Create HTTPS connection pool
118
pool = urllib3.HTTPSConnectionPool('httpbin.org', port=443)
119
120
# Make HTTPS requests
121
resp = pool.urlopen('GET', '/get')
122
print(f"HTTPS response: {resp.status}")
123
124
pool.close()
125
```
126
127
### Connection Pool with Custom Configuration
128
129
```python
130
import urllib3
131
132
# Pool with custom settings
133
pool = urllib3.HTTPSConnectionPool(
134
'api.example.com',
135
port=443,
136
maxsize=10, # Up to 10 concurrent connections
137
block=True, # Block when pool is full
138
timeout=urllib3.Timeout(connect=2.0, read=5.0),
139
retries=urllib3.Retry(total=3, backoff_factor=0.3),
140
headers={'User-Agent': 'MyApp/1.0'}
141
)
142
143
# Make requests with the configured defaults
144
resp = pool.urlopen('GET', '/api/data')
145
```
146
147
### SSL/TLS Configuration
148
149
```python
150
import urllib3
151
import ssl
152
153
# Custom SSL context
154
ctx = ssl.create_default_context()
155
ctx.check_hostname = True
156
ctx.verify_mode = ssl.CERT_REQUIRED
157
158
# HTTPS pool with custom SSL context
159
pool = urllib3.HTTPSConnectionPool(
160
'secure.example.com',
161
ssl_context=ctx
162
)
163
164
resp = pool.urlopen('GET', '/secure-endpoint')
165
```
166
167
### Client Certificate Authentication
168
169
```python
170
import urllib3
171
172
# HTTPS pool with client certificate
173
pool = urllib3.HTTPSConnectionPool(
174
'client-cert.example.com',
175
cert_file='/path/to/client.crt',
176
key_file='/path/to/client.key',
177
ca_certs='/path/to/ca-bundle.crt'
178
)
179
180
resp = pool.urlopen('GET', '/authenticated-endpoint')
181
```
182
183
### Connection Pool from URL
184
185
```python
186
import urllib3
187
188
# Create pool from URL
189
pool = urllib3.connection_from_url('https://api.example.com:8443')
190
191
# Use the pool
192
resp = pool.urlopen('GET', '/api/v1/data')
193
print(f"Response: {resp.status}")
194
195
pool.close()
196
```
197
198
### Pool Lifecycle Management
199
200
```python
201
import urllib3
202
203
# Context manager usage (recommended)
204
with urllib3.HTTPSConnectionPool('api.example.com') as pool:
205
resp1 = pool.urlopen('GET', '/endpoint1')
206
resp2 = pool.urlopen('GET', '/endpoint2')
207
# Pool is automatically closed when exiting the context
208
209
# Manual lifecycle management
210
pool = urllib3.HTTPConnectionPool('api.example.com')
211
try:
212
resp = pool.urlopen('GET', '/data')
213
# Process response
214
finally:
215
pool.close() # Always close the pool
216
```
217
218
### Advanced Connection Pool Settings
219
220
```python
221
import urllib3
222
223
# Pool with comprehensive configuration
224
pool = urllib3.HTTPSConnectionPool(
225
'api.example.com',
226
227
# Connection pool settings
228
maxsize=20, # Max connections in pool
229
block=True, # Block when pool full
230
231
# Timeout configuration
232
timeout=urllib3.Timeout(
233
connect=2.0, # Connection timeout
234
read=10.0 # Read timeout
235
),
236
237
# Retry configuration
238
retries=urllib3.Retry(
239
total=3, # Total retry attempts
240
connect=2, # Connection retry attempts
241
read=2, # Read retry attempts
242
backoff_factor=0.3 # Exponential backoff
243
),
244
245
# SSL/TLS settings
246
cert_reqs='CERT_REQUIRED', # Require valid certificate
247
ca_certs='/etc/ssl/certs/ca-certificates.crt',
248
249
# Default headers
250
headers={
251
'User-Agent': 'MyApp/2.0',
252
'Accept': 'application/json'
253
}
254
)
255
256
# All requests use these default settings
257
resp = pool.urlopen('GET', '/api/data')
258
```
259
260
## Connection Pool Behavior
261
262
### Connection Reuse
263
- Connections are automatically reused for multiple requests
264
- Connections are returned to the pool after each request
265
- Pool maintains connections until they expire or are closed
266
267
### Thread Safety
268
- Connection pools are thread-safe
269
- Multiple threads can safely share the same pool
270
- Connections are properly synchronized
271
272
### Error Handling
273
- Failed connections are automatically removed from pool
274
- New connections are created as needed
275
- Configurable retry behavior for various failure types
276
277
### Resource Management
278
- Pools should be closed when no longer needed
279
- Use context managers for automatic cleanup
280
- Monitor pool usage to avoid resource leaks