0
# Exception Handling
1
2
Comprehensive exception hierarchy for handling various error conditions that can occur during HTTP requests. The requests library provides specific exceptions for different types of errors, allowing for precise error handling.
3
4
## Capabilities
5
6
### Base Exception
7
8
All requests exceptions inherit from the base RequestException class.
9
10
```python { .api }
11
class RequestException(IOError):
12
"""
13
Base exception for all request-related errors.
14
15
Attributes:
16
- response: Response object (if available)
17
- request: Request object that caused the exception
18
"""
19
20
def __init__(self, *args, **kwargs):
21
"""
22
Initialize RequestException.
23
24
Parameters:
25
- response: Response object (optional)
26
- request: Request object (optional)
27
"""
28
```
29
30
### HTTP Errors
31
32
Exceptions related to HTTP status codes and server responses.
33
34
```python { .api }
35
class HTTPError(RequestException):
36
"""
37
An HTTP error occurred.
38
39
Raised by Response.raise_for_status() for 4xx and 5xx status codes.
40
"""
41
42
class InvalidJSONError(RequestException):
43
"""A JSON error occurred."""
44
45
class JSONDecodeError(InvalidJSONError):
46
"""Couldn't decode the text into JSON."""
47
```
48
49
### Connection Errors
50
51
Exceptions related to network connectivity and connection issues.
52
53
```python { .api }
54
class ConnectionError(RequestException):
55
"""A connection error occurred."""
56
57
class ProxyError(ConnectionError):
58
"""A proxy error occurred."""
59
60
class SSLError(ConnectionError):
61
"""An SSL error occurred."""
62
```
63
64
### Timeout Errors
65
66
Exceptions related to request timeouts.
67
68
```python { .api }
69
class Timeout(RequestException):
70
"""
71
The request timed out.
72
73
Catching this error will catch both ConnectTimeout and ReadTimeout.
74
"""
75
76
class ConnectTimeout(ConnectionError, Timeout):
77
"""
78
The request timed out while trying to connect to the remote server.
79
80
Requests that produced this error are safe to retry.
81
"""
82
83
class ReadTimeout(Timeout):
84
"""The server did not send any data in the allotted amount of time."""
85
```
86
87
### URL and Request Errors
88
89
Exceptions related to malformed URLs and invalid requests.
90
91
```python { .api }
92
class URLRequired(RequestException):
93
"""A valid URL is required to make a request."""
94
95
class MissingSchema(RequestException, ValueError):
96
"""The URL scheme (e.g. http or https) is missing."""
97
98
class InvalidSchema(RequestException, ValueError):
99
"""The URL scheme is invalid."""
100
101
class InvalidURL(RequestException, ValueError):
102
"""The URL provided is invalid."""
103
104
class InvalidHeader(RequestException, ValueError):
105
"""The header provided is invalid."""
106
107
class InvalidProxyURL(InvalidURL):
108
"""The proxy URL provided is invalid."""
109
```
110
111
### Redirect Errors
112
113
Exceptions related to HTTP redirects.
114
115
```python { .api }
116
class TooManyRedirects(RequestException):
117
"""Too many redirects occurred."""
118
```
119
120
### Content and Encoding Errors
121
122
Exceptions related to response content processing.
123
124
```python { .api }
125
class ChunkedEncodingError(RequestException):
126
"""The server declared chunked encoding but sent an invalid chunk."""
127
128
class ContentDecodingError(RequestException):
129
"""Failed to decode response content."""
130
131
class StreamConsumedError(RequestException, TypeError):
132
"""The content for this response was already consumed."""
133
134
class RetryError(RequestException):
135
"""Custom retries logic failed."""
136
137
class UnrewindableBodyError(RequestException):
138
"""The request body cannot be rewound for a retry."""
139
```
140
141
### Warning Classes
142
143
Warning classes for non-fatal issues.
144
145
```python { .api }
146
class RequestsWarning(Warning):
147
"""Base warning for requests-related warnings."""
148
149
class FileModeWarning(RequestsWarning, DeprecationWarning):
150
"""Warning for file mode issues."""
151
152
class RequestsDependencyWarning(RequestsWarning):
153
"""Warning for dependency-related issues."""
154
```
155
156
## Usage Examples
157
158
### Basic Exception Handling
159
160
```python
161
import requests
162
from requests.exceptions import RequestException, HTTPError, ConnectionError, Timeout
163
164
try:
165
response = requests.get('https://api.example.com/data', timeout=5)
166
response.raise_for_status() # Raises HTTPError for bad status codes
167
data = response.json()
168
except HTTPError as e:
169
print(f"HTTP error occurred: {e}")
170
print(f"Status code: {e.response.status_code}")
171
except ConnectionError as e:
172
print(f"Connection error occurred: {e}")
173
except Timeout as e:
174
print(f"Request timed out: {e}")
175
except RequestException as e:
176
print(f"An error occurred: {e}")
177
```
178
179
### Specific Exception Handling
180
181
```python
182
import requests
183
from requests.exceptions import (
184
ConnectTimeout, ReadTimeout, SSLError,
185
JSONDecodeError, TooManyRedirects
186
)
187
188
def robust_api_call(url, max_retries=3):
189
for attempt in range(max_retries):
190
try:
191
response = requests.get(url, timeout=(5, 10)) # (connect, read) timeout
192
response.raise_for_status()
193
return response.json()
194
195
except ConnectTimeout:
196
print(f"Connection timeout on attempt {attempt + 1}")
197
if attempt == max_retries - 1:
198
raise
199
200
except ReadTimeout:
201
print(f"Read timeout on attempt {attempt + 1}")
202
if attempt == max_retries - 1:
203
raise
204
205
except SSLError as e:
206
print(f"SSL error: {e}")
207
# SSL errors usually shouldn't be retried
208
raise
209
210
except JSONDecodeError:
211
print("Response is not valid JSON")
212
# Try to return raw text instead
213
return response.text
214
215
except TooManyRedirects:
216
print("Too many redirects")
217
raise
218
219
except HTTPError as e:
220
if e.response.status_code >= 500:
221
# Server errors might be temporary, retry
222
print(f"Server error {e.response.status_code} on attempt {attempt + 1}")
223
if attempt == max_retries - 1:
224
raise
225
else:
226
# Client errors usually shouldn't be retried
227
print(f"Client error {e.response.status_code}")
228
raise
229
230
result = robust_api_call('https://api.example.com/data')
231
```
232
233
### Connection Error Handling
234
235
```python
236
import requests
237
from requests.exceptions import ConnectionError, ProxyError, SSLError
238
239
def handle_connection_errors(url):
240
try:
241
response = requests.get(url)
242
return response
243
244
except ProxyError as e:
245
print(f"Proxy error: {e}")
246
# Try without proxy
247
return requests.get(url, proxies={'http': '', 'https': ''})
248
249
except SSLError as e:
250
print(f"SSL error: {e}")
251
# Option 1: Use HTTP instead (not recommended for production)
252
# return requests.get(url.replace('https://', 'http://'))
253
254
# Option 2: Disable SSL verification (not recommended for production)
255
# return requests.get(url, verify=False)
256
257
# Option 3: Provide custom CA bundle
258
return requests.get(url, verify='/path/to/ca-bundle.crt')
259
260
except ConnectionError as e:
261
print(f"Connection error: {e}")
262
raise # Re-raise if no recovery is possible
263
```
264
265
### URL and Request Error Handling
266
267
```python
268
import requests
269
from requests.exceptions import (
270
URLRequired, MissingSchema, InvalidSchema,
271
InvalidURL, InvalidHeader
272
)
273
274
def validate_and_request(url, headers=None):
275
try:
276
response = requests.get(url, headers=headers)
277
return response
278
279
except URLRequired:
280
raise ValueError("URL is required")
281
282
except MissingSchema:
283
# Try adding https://
284
return requests.get(f"https://{url}")
285
286
except InvalidSchema as e:
287
print(f"Invalid URL scheme: {e}")
288
raise ValueError(f"URL must start with http:// or https://")
289
290
except InvalidURL as e:
291
print(f"Invalid URL: {e}")
292
raise ValueError("Please provide a valid URL")
293
294
except InvalidHeader as e:
295
print(f"Invalid header: {e}")
296
raise ValueError("Please check header format")
297
298
# Example usage
299
try:
300
response = validate_and_request("example.com") # Missing schema
301
except ValueError as e:
302
print(f"Validation error: {e}")
303
```
304
305
### JSON Error Handling
306
307
```python
308
import requests
309
from requests.exceptions import JSONDecodeError
310
311
def safe_json_request(url):
312
try:
313
response = requests.get(url)
314
response.raise_for_status()
315
316
# Try to parse JSON
317
return response.json()
318
319
except JSONDecodeError as e:
320
print(f"JSON decode error: {e}")
321
print(f"Response content: {response.text[:200]}...") # First 200 chars
322
323
# Return raw text or None based on content type
324
content_type = response.headers.get('content-type', '')
325
if 'json' in content_type.lower():
326
# Expected JSON but got invalid JSON
327
raise ValueError("Expected JSON response but got invalid JSON")
328
else:
329
# Not JSON, return text
330
return response.text
331
332
data = safe_json_request('https://api.example.com/data')
333
```
334
335
### Comprehensive Error Handling Pattern
336
337
```python
338
import requests
339
from requests.exceptions import RequestException
340
import time
341
342
def reliable_request(url, max_retries=3, backoff_factor=1):
343
"""
344
Make a reliable HTTP request with exponential backoff retry logic.
345
"""
346
last_exception = None
347
348
for attempt in range(max_retries):
349
try:
350
response = requests.get(url, timeout=(5, 30))
351
response.raise_for_status()
352
return response
353
354
except requests.exceptions.ConnectTimeout:
355
print(f"Connect timeout on attempt {attempt + 1}")
356
last_exception = sys.exc_info()[1]
357
358
except requests.exceptions.ReadTimeout:
359
print(f"Read timeout on attempt {attempt + 1}")
360
last_exception = sys.exc_info()[1]
361
362
except requests.exceptions.HTTPError as e:
363
if 500 <= e.response.status_code < 600:
364
# Server error - retry
365
print(f"Server error {e.response.status_code} on attempt {attempt + 1}")
366
last_exception = e
367
else:
368
# Client error - don't retry
369
raise
370
371
except requests.exceptions.ConnectionError:
372
print(f"Connection error on attempt {attempt + 1}")
373
last_exception = sys.exc_info()[1]
374
375
except RequestException as e:
376
# Other request exceptions
377
print(f"Request exception on attempt {attempt + 1}: {e}")
378
last_exception = e
379
380
# Wait before retrying (exponential backoff)
381
if attempt < max_retries - 1:
382
wait_time = backoff_factor * (2 ** attempt)
383
print(f"Waiting {wait_time} seconds before retry...")
384
time.sleep(wait_time)
385
386
# All retries failed
387
raise last_exception
388
389
# Usage
390
try:
391
response = reliable_request('https://api.example.com/unreliable-endpoint')
392
data = response.json()
393
except Exception as e:
394
print(f"Request ultimately failed: {e}")
395
```
396
397
### Exception Information Access
398
399
```python
400
import requests
401
from requests.exceptions import HTTPError, RequestException
402
403
try:
404
response = requests.get('https://httpbin.org/status/404')
405
response.raise_for_status()
406
except HTTPError as e:
407
print(f"HTTP Error: {e}")
408
print(f"Status Code: {e.response.status_code}")
409
print(f"Response Text: {e.response.text}")
410
print(f"Request URL: {e.request.url}")
411
print(f"Request Method: {e.request.method}")
412
except RequestException as e:
413
print(f"Request Exception: {e}")
414
if hasattr(e, 'response') and e.response is not None:
415
print(f"Response Status: {e.response.status_code}")
416
if hasattr(e, 'request') and e.request is not None:
417
print(f"Request URL: {e.request.url}")
418
```
419
420
## Best Practices
421
422
1. **Always catch specific exceptions** before more general ones
423
2. **Use `response.raise_for_status()`** to automatically raise HTTPError for bad status codes
424
3. **Set appropriate timeouts** to avoid hanging requests
425
4. **Implement retry logic** for transient errors (timeouts, 5xx errors)
426
5. **Don't retry client errors** (4xx status codes) as they indicate request problems
427
6. **Log exception details** for debugging
428
7. **Provide meaningful error messages** to users
429
8. **Handle SSL errors carefully** - avoid disabling verification in production