0
# Error Handling and Exceptions
1
2
Complete DB-API 2.0 exception hierarchy providing structured error handling for different types of database and interface errors. The redshift_connector implements a comprehensive exception system that enables applications to handle errors appropriately based on their type and severity.
3
4
## Capabilities
5
6
### Exception Hierarchy
7
8
DB-API 2.0 compliant exception hierarchy with specific error types for different failure scenarios.
9
10
```python { .api }
11
class Warning(Exception):
12
"""
13
Generic exception raised for important database warnings like data truncations.
14
This exception is not currently used.
15
16
This exception is part of the DB-API 2.0 specification.
17
"""
18
19
class Error(Exception):
20
"""
21
Generic exception that is the base exception of all other error exceptions.
22
23
This exception is part of the DB-API 2.0 specification.
24
"""
25
26
class InterfaceError(Error):
27
"""
28
Generic exception raised for errors that are related to the database
29
interface rather than the database itself. For example, if the interface
30
attempts to use an SSL connection but the server refuses, an InterfaceError
31
will be raised.
32
33
This exception is part of the DB-API 2.0 specification.
34
"""
35
36
class DatabaseError(Error):
37
"""
38
Generic exception raised for errors that are related to the database.
39
This exception is currently never raised.
40
41
This exception is part of the DB-API 2.0 specification.
42
"""
43
44
class DataError(DatabaseError):
45
"""
46
Generic exception raised for errors that are due to problems with the
47
processed data. This exception is not currently raised.
48
49
This exception is part of the DB-API 2.0 specification.
50
"""
51
52
class OperationalError(DatabaseError):
53
"""
54
Generic exception raised for errors that are related to the database's
55
operation and not necessarily under the control of the programmer.
56
This exception is currently never raised.
57
58
This exception is part of the DB-API 2.0 specification.
59
"""
60
61
class IntegrityError(DatabaseError):
62
"""
63
Generic exception raised when the relational integrity of the database is
64
affected. This exception is not currently raised.
65
66
This exception is part of the DB-API 2.0 specification.
67
"""
68
69
class InternalError(DatabaseError):
70
"""
71
Generic exception raised when the database encounters an internal error.
72
This is currently only raised when unexpected state occurs in the
73
interface itself, and is typically the result of a interface bug.
74
75
This exception is part of the DB-API 2.0 specification.
76
"""
77
78
class ProgrammingError(DatabaseError):
79
"""
80
Generic exception raised for programming errors. For example, this
81
exception is raised if more parameter fields are in a query string than
82
there are available parameters.
83
84
This exception is part of the DB-API 2.0 specification.
85
"""
86
87
class NotSupportedError(DatabaseError):
88
"""
89
Generic exception raised in case a method or database API was used which
90
is not supported by the database.
91
92
This exception is part of the DB-API 2.0 specification.
93
"""
94
```
95
96
### Array-Specific Exceptions
97
98
Specialized exceptions for array data type operations and validation.
99
100
```python { .api }
101
class ArrayContentNotSupportedError(NotSupportedError):
102
"""
103
Raised when attempting to transmit an array where the base type is not
104
supported for binary data transfer by the interface.
105
"""
106
107
class ArrayContentNotHomogenousError(ProgrammingError):
108
"""
109
Raised when attempting to transmit an array that doesn't contain only a
110
single type of object.
111
"""
112
113
class ArrayDimensionsNotConsistentError(ProgrammingError):
114
"""
115
Raised when attempting to transmit an array that has inconsistent
116
multi-dimension sizes.
117
"""
118
```
119
120
### Error Handling Patterns
121
122
Common error handling patterns and best practices for different scenarios.
123
124
```python
125
import redshift_connector
126
from redshift_connector import (
127
Error, InterfaceError, DatabaseError, ProgrammingError,
128
OperationalError, DataError, NotSupportedError
129
)
130
131
# Basic error handling pattern
132
try:
133
conn = redshift_connector.connect(
134
host='invalid-host.redshift.amazonaws.com',
135
database='dev',
136
user='user',
137
password='password'
138
)
139
except InterfaceError as e:
140
print(f"Connection interface error: {e}")
141
# Handle connection issues (network, SSL, authentication)
142
except Error as e:
143
print(f"General database error: {e}")
144
# Catch-all for other database errors
145
146
# Query execution error handling
147
try:
148
cursor = conn.cursor()
149
cursor.execute("SELECT * FROM nonexistent_table")
150
results = cursor.fetchall()
151
except ProgrammingError as e:
152
print(f"SQL programming error: {e}")
153
# Handle SQL syntax errors, missing tables/columns, etc.
154
except DatabaseError as e:
155
print(f"Database error: {e}")
156
# Handle database-level errors
157
except Error as e:
158
print(f"General error: {e}")
159
160
# Parameter handling errors
161
try:
162
cursor.execute("INSERT INTO table (col1, col2) VALUES (%s, %s)", ('value1',)) # Missing parameter
163
except ProgrammingError as e:
164
print(f"Parameter mismatch: {e}")
165
166
# Array handling errors
167
try:
168
cursor.execute("INSERT INTO array_table (arr_col) VALUES (%s)", ([[1, 2], [3, 4, 5]])) # Inconsistent dimensions
169
except ArrayDimensionsNotConsistentError as e:
170
print(f"Array dimension error: {e}")
171
172
try:
173
cursor.execute("INSERT INTO array_table (arr_col) VALUES (%s)", ([1, 'mixed', 3.14])) # Mixed types
174
except ArrayContentNotHomogenousError as e:
175
print(f"Array content error: {e}")
176
177
# Data science integration errors
178
try:
179
df = cursor.fetch_dataframe()
180
except InterfaceError as e:
181
if "pandas" in str(e):
182
print("pandas not available. Install with: pip install redshift_connector[full]")
183
elif "numpy" in str(e):
184
print("numpy not available. Install with: pip install redshift_connector[full]")
185
186
# Authentication errors
187
try:
188
conn = redshift_connector.connect(
189
iam=True,
190
ssl=False, # Invalid: IAM requires SSL
191
cluster_identifier='cluster'
192
)
193
except InterfaceError as e:
194
print(f"Authentication configuration error: {e}")
195
196
# Connection cleanup with error handling
197
try:
198
# Database operations
199
cursor = conn.cursor()
200
cursor.execute("SELECT 1")
201
result = cursor.fetchone()
202
except DatabaseError as e:
203
print(f"Database operation failed: {e}")
204
finally:
205
# Always clean up connections
206
if 'cursor' in locals():
207
cursor.close()
208
if 'conn' in locals():
209
conn.close()
210
```
211
212
### Context Manager Error Handling
213
214
Using context managers for automatic resource cleanup with proper error handling.
215
216
```python { .api }
217
# Connection context manager with error handling
218
try:
219
with redshift_connector.connect(
220
host='examplecluster.abc123xyz789.us-west-1.redshift.amazonaws.com',
221
database='dev',
222
user='awsuser',
223
password='password'
224
) as conn:
225
with conn.cursor() as cursor:
226
cursor.execute("SELECT COUNT(*) FROM large_table")
227
count = cursor.fetchone()[0]
228
print(f"Table has {count} rows")
229
230
except InterfaceError as e:
231
print(f"Connection or interface error: {e}")
232
except ProgrammingError as e:
233
print(f"SQL or programming error: {e}")
234
except DatabaseError as e:
235
print(f"Database error: {e}")
236
# Connection and cursor automatically closed even if exceptions occur
237
```
238
239
### Transaction Error Handling
240
241
Error handling patterns for transaction management and rollback scenarios.
242
243
```python
244
import redshift_connector
245
from redshift_connector import DatabaseError, ProgrammingError
246
247
conn = redshift_connector.connect(...)
248
249
try:
250
# Start transaction (autocommit=False is default)
251
cursor = conn.cursor()
252
253
# Perform multiple operations
254
cursor.execute("INSERT INTO orders (customer_id, amount) VALUES (%s, %s)", (123, 99.99))
255
cursor.execute("UPDATE inventory SET quantity = quantity - 1 WHERE product_id = %s", (456,))
256
cursor.execute("INSERT INTO audit_log (action, timestamp) VALUES (%s, %s)", ('order_placed', '2023-01-01'))
257
258
# Commit transaction
259
conn.commit()
260
print("Transaction completed successfully")
261
262
except ProgrammingError as e:
263
print(f"SQL error in transaction: {e}")
264
conn.rollback()
265
print("Transaction rolled back")
266
267
except DatabaseError as e:
268
print(f"Database error in transaction: {e}")
269
conn.rollback()
270
print("Transaction rolled back")
271
272
except Exception as e:
273
print(f"Unexpected error: {e}")
274
conn.rollback()
275
print("Transaction rolled back")
276
277
finally:
278
cursor.close()
279
conn.close()
280
```
281
282
### Authentication Error Handling
283
284
Specific error handling for various authentication scenarios and configurations.
285
286
```python
287
import redshift_connector
288
from redshift_connector import InterfaceError
289
290
# IAM authentication error handling
291
try:
292
conn = redshift_connector.connect(
293
iam=True,
294
cluster_identifier='my-cluster',
295
db_user='testuser',
296
access_key_id='invalid_key',
297
secret_access_key='invalid_secret',
298
region='us-west-2'
299
)
300
except InterfaceError as e:
301
if "SSL must be enabled when using IAM" in str(e):
302
print("IAM authentication requires SSL to be enabled")
303
elif "Invalid connection property" in str(e):
304
print("Invalid authentication configuration")
305
elif "credentials" in str(e).lower():
306
print("Invalid AWS credentials")
307
else:
308
print(f"Authentication error: {e}")
309
310
# Identity provider error handling
311
try:
312
conn = redshift_connector.connect(
313
credentials_provider='BrowserIdcAuthPlugin',
314
iam=True # Invalid: IdC plugins cannot be used with IAM
315
)
316
except InterfaceError as e:
317
if "can not use this authentication plugin with IAM enabled" in str(e):
318
print("Identity Center plugins cannot be used with IAM=True")
319
320
# SSL configuration error handling
321
try:
322
conn = redshift_connector.connect(
323
credentials_provider='BrowserAzureOAuth2CredentialsProvider',
324
ssl=False # Invalid: Authentication plugins require SSL
325
)
326
except InterfaceError as e:
327
if "Authentication must use an SSL connection" in str(e):
328
print("Authentication plugins require SSL to be enabled")
329
```
330
331
### Logging and Debugging
332
333
Error logging and debugging utilities for troubleshooting connection and query issues.
334
335
```python { .api }
336
import logging
337
import redshift_connector
338
from redshift_connector.utils import make_divider_block, mask_secure_info_in_props
339
340
# Enable logging for debugging
341
logging.basicConfig(level=logging.DEBUG)
342
logger = logging.getLogger('redshift_connector')
343
344
try:
345
conn = redshift_connector.connect(
346
host='examplecluster.abc123xyz789.us-west-1.redshift.amazonaws.com',
347
database='dev',
348
user='awsuser',
349
password='password'
350
)
351
# Connection details are automatically logged (with sensitive info masked)
352
353
except Exception as e:
354
logger.error(f"Connection failed: {e}")
355
# Detailed error information available in logs
356
357
# Utility functions for secure logging
358
def log_connection_attempt(conn_params):
359
"""Log connection attempt with sensitive information masked."""
360
from redshift_connector.redshift_property import RedshiftProperty
361
362
info = RedshiftProperty()
363
for key, value in conn_params.items():
364
info.put(key, value)
365
366
masked_info = mask_secure_info_in_props(info)
367
logger.info(make_divider_block())
368
logger.info("Connection attempt with parameters:")
369
logger.info(str(masked_info))
370
logger.info(make_divider_block())
371
```
372
373
### Recovery and Retry Patterns
374
375
Patterns for handling transient errors and implementing retry logic.
376
377
```python
378
import time
379
import redshift_connector
380
from redshift_connector import OperationalError, InterfaceError
381
382
def connect_with_retry(max_attempts=3, retry_delay=1, **conn_params):
383
"""
384
Attempt connection with exponential backoff retry logic.
385
"""
386
for attempt in range(max_attempts):
387
try:
388
conn = redshift_connector.connect(**conn_params)
389
return conn
390
except (OperationalError, InterfaceError) as e:
391
if attempt == max_attempts - 1:
392
raise # Re-raise on final attempt
393
394
print(f"Connection attempt {attempt + 1} failed: {e}")
395
time.sleep(retry_delay * (2 ** attempt)) # Exponential backoff
396
397
def execute_with_retry(cursor, sql, params=None, max_attempts=3):
398
"""
399
Execute query with retry logic for transient errors.
400
"""
401
for attempt in range(max_attempts):
402
try:
403
if params:
404
cursor.execute(sql, params)
405
else:
406
cursor.execute(sql)
407
return cursor.fetchall()
408
except OperationalError as e:
409
if "timeout" in str(e).lower() and attempt < max_attempts - 1:
410
print(f"Query timeout, retrying... (attempt {attempt + 1})")
411
time.sleep(2 ** attempt)
412
continue
413
raise
414
except DatabaseError as e:
415
# Don't retry programming errors or constraint violations
416
raise
417
418
# Usage example
419
try:
420
conn = connect_with_retry(
421
host='examplecluster.abc123xyz789.us-west-1.redshift.amazonaws.com',
422
database='dev',
423
user='awsuser',
424
password='password',
425
max_attempts=3
426
)
427
428
cursor = conn.cursor()
429
results = execute_with_retry(cursor, "SELECT COUNT(*) FROM large_table")
430
print(f"Query result: {results}")
431
432
except Exception as e:
433
print(f"Failed after retries: {e}")
434
finally:
435
if 'cursor' in locals():
436
cursor.close()
437
if 'conn' in locals():
438
conn.close()
439
```