0
# Response Handling
1
2
Comprehensive HTTP response objects with content decoding, streaming support, metadata access, and flexible data consumption patterns. urllib3 provides rich response handling capabilities for various use cases.
3
4
## Capabilities
5
6
### BaseHTTPResponse
7
8
Base interface for all HTTP response objects, providing core functionality for accessing response data, headers, and metadata.
9
10
```python { .api }
11
class BaseHTTPResponse:
12
@property
13
def status(self) -> int:
14
"""HTTP status code (200, 404, 500, etc.)"""
15
16
@property
17
def headers(self) -> HTTPHeaderDict:
18
"""Response headers as case-insensitive dictionary"""
19
20
@property
21
def data(self) -> bytes:
22
"""Complete response body as bytes"""
23
24
def read(self, amt=None, decode_content=None) -> bytes:
25
"""
26
Read response data.
27
28
Parameters:
29
- amt: Maximum bytes to read (None for all)
30
- decode_content: Whether to decode compressed content
31
32
Returns:
33
bytes: Response data
34
"""
35
36
def stream(self, amt=None, decode_content=None):
37
"""
38
Stream response data in chunks.
39
40
Parameters:
41
- amt: Chunk size in bytes
42
- decode_content: Whether to decode compressed content
43
44
Yields:
45
bytes: Chunks of response data
46
"""
47
48
def close(self):
49
"""Close the response and release connection back to pool"""
50
51
def isclosed(self) -> bool:
52
"""Check if response has been closed"""
53
```
54
55
### HTTPResponse
56
57
Full HTTP response implementation with advanced features including automatic content decoding, JSON parsing, and comprehensive metadata access.
58
59
```python { .api }
60
class HTTPResponse(BaseHTTPResponse):
61
@property
62
def reason(self) -> str:
63
"""HTTP reason phrase (OK, Not Found, etc.)"""
64
65
@property
66
def version(self) -> int:
67
"""HTTP version (11 for HTTP/1.1, 20 for HTTP/2)"""
68
69
@property
70
def connection(self):
71
"""Underlying connection object"""
72
73
@property
74
def retries(self) -> Retry:
75
"""Retry configuration that was used"""
76
77
@property
78
def url(self) -> str:
79
"""Final URL after redirects"""
80
81
def json(self) -> any:
82
"""
83
Parse response body as JSON.
84
85
Returns:
86
any: Parsed JSON data
87
88
Raises:
89
json.JSONDecodeError: If response is not valid JSON
90
"""
91
92
def getheader(self, name: str, default=None) -> str:
93
"""
94
Get single header value.
95
96
Parameters:
97
- name: Header name (case-insensitive)
98
- default: Default value if header not found
99
100
Returns:
101
str: Header value or default
102
"""
103
104
def getheaders(self, name: str) -> list[str]:
105
"""
106
Get all values for a header (for multi-value headers).
107
108
Parameters:
109
- name: Header name (case-insensitive)
110
111
Returns:
112
list[str]: List of header values
113
"""
114
```
115
116
### HTTPHeaderDict
117
118
Case-insensitive dictionary for HTTP headers with support for multi-value headers.
119
120
```python { .api }
121
class HTTPHeaderDict:
122
def __init__(self, headers=None):
123
"""
124
Case-insensitive HTTP header dictionary.
125
126
Parameters:
127
- headers: Initial headers (dict, list of tuples, or HTTPHeaderDict)
128
"""
129
130
def add(self, key: str, val: str):
131
"""Add header value (supports multiple values per header)"""
132
133
def extend(self, other):
134
"""Extend headers from another dict/HTTPHeaderDict"""
135
136
def get(self, key: str, default=None) -> str:
137
"""Get header value (case-insensitive)"""
138
139
def get_all(self, key: str) -> list[str]:
140
"""Get all values for header"""
141
142
def items(self):
143
"""Iterate over (name, value) pairs"""
144
145
def values(self):
146
"""Iterate over header values"""
147
148
def keys(self):
149
"""Iterate over header names"""
150
```
151
152
## Usage Examples
153
154
### Basic Response Handling
155
156
```python
157
import urllib3
158
159
http = urllib3.PoolManager()
160
resp = http.request('GET', 'https://httpbin.org/json')
161
162
# Access response properties
163
print(f"Status: {resp.status}") # 200
164
print(f"Reason: {resp.reason}") # OK
165
print(f"Headers: {resp.headers}") # HTTPHeaderDict
166
print(f"Content-Type: {resp.headers['content-type']}")
167
print(f"Data: {resp.data.decode('utf-8')}") # Response body as string
168
169
# Close response to return connection to pool
170
resp.close()
171
```
172
173
### JSON Response Handling
174
175
```python
176
import urllib3
177
178
http = urllib3.PoolManager()
179
resp = http.request('GET', 'https://jsonplaceholder.typicode.com/posts/1')
180
181
# Parse JSON response
182
data = resp.json()
183
print(f"Title: {data['title']}")
184
print(f"Body: {data['body']}")
185
186
resp.close()
187
```
188
189
### Streaming Large Responses
190
191
```python
192
import urllib3
193
194
http = urllib3.PoolManager()
195
resp = http.request('GET', 'https://httpbin.org/stream/1000',
196
preload_content=False) # Don't load entire response
197
198
# Stream response in chunks
199
total_bytes = 0
200
for chunk in resp.stream(1024): # 1KB chunks
201
total_bytes += len(chunk)
202
print(f"Received {len(chunk)} bytes, total: {total_bytes}")
203
204
print(f"Total downloaded: {total_bytes} bytes")
205
resp.close()
206
```
207
208
### Reading Response in Chunks
209
210
```python
211
import urllib3
212
213
http = urllib3.PoolManager()
214
resp = http.request('GET', 'https://httpbin.org/bytes/10000',
215
preload_content=False)
216
217
# Read specific amounts
218
chunk1 = resp.read(1024) # Read first 1KB
219
chunk2 = resp.read(2048) # Read next 2KB
220
remaining = resp.read() # Read rest
221
222
print(f"Chunk 1: {len(chunk1)} bytes")
223
print(f"Chunk 2: {len(chunk2)} bytes")
224
print(f"Remaining: {len(remaining)} bytes")
225
226
resp.close()
227
```
228
229
### Header Manipulation
230
231
```python
232
import urllib3
233
234
http = urllib3.PoolManager()
235
resp = http.request('GET', 'https://httpbin.org/response-headers',
236
fields={'Content-Type': 'application/json',
237
'Custom-Header': 'MyValue'})
238
239
# Access headers in various ways
240
print(f"Content-Type: {resp.headers['content-type']}")
241
print(f"Content-Type: {resp.getheader('content-type')}")
242
243
# Check if header exists
244
if 'custom-header' in resp.headers:
245
print(f"Custom header: {resp.headers['custom-header']}")
246
247
# Iterate over all headers
248
for name, value in resp.headers.items():
249
print(f"{name}: {value}")
250
251
resp.close()
252
```
253
254
### Working with Multi-Value Headers
255
256
```python
257
import urllib3
258
259
# Create HTTPHeaderDict
260
headers = urllib3.HTTPHeaderDict()
261
headers.add('Set-Cookie', 'session=abc123; Path=/')
262
headers.add('Set-Cookie', 'theme=dark; Path=/')
263
264
# Get all values for a header
265
cookies = headers.get_all('set-cookie')
266
print(f"All cookies: {cookies}")
267
268
# Get first value only
269
first_cookie = headers.get('set-cookie')
270
print(f"First cookie: {first_cookie}")
271
```
272
273
### Response Context Management
274
275
```python
276
import urllib3
277
278
http = urllib3.PoolManager()
279
280
# Using context manager (recommended)
281
with http.request('GET', 'https://httpbin.org/json') as resp:
282
data = resp.json()
283
print(f"Status: {resp.status}")
284
print(f"Data: {data}")
285
# Response is automatically closed
286
287
# Manual management
288
resp = http.request('GET', 'https://httpbin.org/json')
289
try:
290
data = resp.json()
291
# Process data
292
finally:
293
resp.close() # Always close
294
```
295
296
### Handling Different Content Types
297
298
```python
299
import urllib3
300
301
http = urllib3.PoolManager()
302
303
# JSON response
304
json_resp = http.request('GET', 'https://httpbin.org/json')
305
json_data = json_resp.json()
306
307
# Text response
308
text_resp = http.request('GET', 'https://httpbin.org/html')
309
html_content = text_resp.data.decode('utf-8')
310
311
# Binary response
312
binary_resp = http.request('GET', 'https://httpbin.org/bytes/1000')
313
binary_data = binary_resp.data # Raw bytes
314
315
# Close all responses
316
json_resp.close()
317
text_resp.close()
318
binary_resp.close()
319
```
320
321
### Error Response Handling
322
323
```python
324
import urllib3
325
326
http = urllib3.PoolManager()
327
328
resp = http.request('GET', 'https://httpbin.org/status/404')
329
330
if resp.status == 404:
331
print("Resource not found")
332
elif resp.status >= 400:
333
print(f"Client error: {resp.status} {resp.reason}")
334
elif resp.status >= 500:
335
print(f"Server error: {resp.status} {resp.reason}")
336
else:
337
print("Success!")
338
339
# Response data might contain error details
340
error_body = resp.data.decode('utf-8')
341
print(f"Error response: {error_body}")
342
343
resp.close()
344
```
345
346
## Content Decoding
347
348
urllib3 automatically handles common content encodings:
349
350
- **gzip**: Automatic decompression of gzip-encoded content
351
- **deflate**: Automatic decompression of deflate-encoded content
352
- **brotli**: Automatic decompression of brotli-encoded content (if available)
353
- **zstd**: Automatic decompression of zstd-encoded content (if available)
354
355
Content decoding can be controlled with the `decode_content` parameter in request methods.