0
# Exception Handling
1
2
Comprehensive exception hierarchy providing detailed error information and compatibility with DB-API 2.0 exception model for robust error handling in ClickHouse applications.
3
4
## Capabilities
5
6
### Base Exception Classes
7
8
Foundation exception classes providing the core error handling framework for all ClickHouse Connect operations.
9
10
```python { .api }
11
class ClickHouseError(Exception):
12
"""
13
Base exception class for all ClickHouse-related errors.
14
15
All ClickHouse Connect exceptions inherit from this base class,
16
allowing for broad exception handling patterns.
17
"""
18
pass
19
20
class Error(Exception):
21
"""
22
Base class for all non-warning database errors.
23
24
Compatible with DB-API 2.0 exception hierarchy.
25
Serves as the root for all error conditions that
26
should halt program execution.
27
"""
28
pass
29
30
class Warning(Exception):
31
"""
32
Exception for important warnings.
33
34
Used for non-fatal issues that should be brought
35
to the attention of the application developer.
36
"""
37
pass
38
```
39
40
### Database Error Categories
41
42
DB-API 2.0 compatible exception hierarchy providing standardized error categorization for different types of database-related failures.
43
44
```python { .api }
45
class InterfaceError(Error):
46
"""
47
Exception for database interface errors.
48
49
Raised when there are problems with the database interface
50
itself rather than the database. Examples include:
51
- Connection configuration errors
52
- Protocol version mismatches
53
- Client library issues
54
"""
55
pass
56
57
class DatabaseError(Error):
58
"""
59
Base exception for database-related errors.
60
61
Parent class for all exceptions that are related to
62
the database operation itself. All other database
63
exception types inherit from this class.
64
"""
65
pass
66
67
class DataError(DatabaseError):
68
"""
69
Exception for data processing errors.
70
71
Raised when there are problems processing data values:
72
- Type conversion failures
73
- Invalid data formats
74
- Data truncation issues
75
- Encoding/decoding problems
76
"""
77
pass
78
79
class OperationalError(DatabaseError):
80
"""
81
Exception for database operation errors.
82
83
Raised for errors related to database operations
84
rather than application programming errors:
85
- Connection failures
86
- Network timeouts
87
- Server unavailable
88
- Transaction conflicts
89
- Memory/resource limits exceeded
90
"""
91
pass
92
93
class IntegrityError(DatabaseError):
94
"""
95
Exception for referential integrity errors.
96
97
Raised when database relational integrity is affected:
98
- Constraint violations
99
- Duplicate key errors
100
- Foreign key constraint failures
101
102
Note: ClickHouse has limited constraint support,
103
so this is mainly for compatibility.
104
"""
105
pass
106
107
class InternalError(DatabaseError):
108
"""
109
Exception for database internal errors.
110
111
Raised when the database encounters internal errors:
112
- Internal database server errors
113
- Corruption issues
114
- System resource failures
115
"""
116
pass
117
118
class ProgrammingError(DatabaseError):
119
"""
120
Exception for programming errors.
121
122
Raised for errors in the application's use of the database:
123
- SQL syntax errors
124
- Invalid table/column names
125
- Wrong number of parameters
126
- Incorrect parameter types
127
- Invalid operation for current state
128
"""
129
pass
130
131
class NotSupportedError(DatabaseError):
132
"""
133
Exception for unsupported operations.
134
135
Raised when attempting to use features not supported
136
by ClickHouse or the current configuration:
137
- Unsupported SQL features
138
- Unsupported data types
139
- Operations not available in current version
140
"""
141
pass
142
```
143
144
### Streaming-Specific Exceptions
145
146
Specialized exceptions for handling errors in streaming operations and data transfer scenarios.
147
148
```python { .api }
149
class StreamClosedError(ClickHouseError):
150
"""
151
Exception for operations on closed streams.
152
153
Raised when attempting to perform operations on
154
a stream that has already been closed or finalized:
155
- Reading from closed result stream
156
- Writing to closed insert stream
157
- Accessing stream after connection closed
158
"""
159
pass
160
161
class StreamFailureError(ClickHouseError):
162
"""
163
Exception for unexpected stream failures.
164
165
Raised when stream operations fail due to
166
unexpected conditions:
167
- Network interruption during streaming
168
- Server-side stream termination
169
- Buffer overflow conditions
170
- Compression/decompression failures
171
"""
172
pass
173
```
174
175
### Connection and Authentication Errors
176
177
Exceptions specifically related to connection establishment, authentication, and network-level communication issues.
178
179
```python { .api }
180
class ConnectionError(OperationalError):
181
"""
182
Exception for connection-related errors.
183
184
Raised when connection to ClickHouse server fails:
185
- Cannot connect to host/port
186
- Connection refused
187
- DNS resolution failures
188
- SSL/TLS handshake failures
189
"""
190
pass
191
192
class AuthenticationError(OperationalError):
193
"""
194
Exception for authentication failures.
195
196
Raised when authentication with ClickHouse server fails:
197
- Invalid username/password
198
- Expired access tokens
199
- Insufficient permissions
200
- Authentication method not supported
201
"""
202
pass
203
204
class TimeoutError(OperationalError):
205
"""
206
Exception for timeout conditions.
207
208
Raised when operations exceed configured timeout limits:
209
- Connection timeout
210
- Query execution timeout
211
- Network read/write timeout
212
"""
213
pass
214
```
215
216
## Usage Examples
217
218
### Basic Exception Handling
219
220
```python
221
import clickhouse_connect
222
from clickhouse_connect.driver.exceptions import (
223
ClickHouseError,
224
ProgrammingError,
225
OperationalError,
226
DataError
227
)
228
229
client = clickhouse_connect.create_client(host='localhost')
230
231
try:
232
# This might raise various exceptions
233
result = client.query("SELECT * FROM non_existent_table")
234
235
except ProgrammingError as e:
236
print(f"SQL error: {e}")
237
# Handle syntax errors, invalid table names, etc.
238
239
except OperationalError as e:
240
print(f"Operation failed: {e}")
241
# Handle connection issues, timeouts, server errors
242
243
except DataError as e:
244
print(f"Data processing error: {e}")
245
# Handle type conversion, format issues
246
247
except ClickHouseError as e:
248
print(f"ClickHouse error: {e}")
249
# Catch any other ClickHouse-specific errors
250
251
except Exception as e:
252
print(f"Unexpected error: {e}")
253
# Handle any other unexpected errors
254
255
finally:
256
client.close()
257
```
258
259
### Connection Error Handling
260
261
```python
262
import clickhouse_connect
263
from clickhouse_connect.driver.exceptions import (
264
ConnectionError,
265
AuthenticationError,
266
TimeoutError,
267
OperationalError
268
)
269
270
def create_robust_client(host, max_retries=3):
271
"""Create client with retry logic for connection issues."""
272
273
for attempt in range(max_retries):
274
try:
275
client = clickhouse_connect.create_client(
276
host=host,
277
connect_timeout=10,
278
send_receive_timeout=60
279
)
280
281
# Test connection
282
client.ping()
283
return client
284
285
except ConnectionError as e:
286
print(f"Connection attempt {attempt + 1} failed: {e}")
287
if attempt == max_retries - 1:
288
raise
289
time.sleep(2 ** attempt) # Exponential backoff
290
291
except AuthenticationError as e:
292
print(f"Authentication failed: {e}")
293
# Don't retry authentication errors
294
raise
295
296
except TimeoutError as e:
297
print(f"Connection timeout: {e}")
298
if attempt == max_retries - 1:
299
raise
300
time.sleep(1)
301
302
raise OperationalError("Failed to establish connection after all retries")
303
304
# Usage
305
try:
306
client = create_robust_client('clickhouse.example.com')
307
print("Connection established successfully")
308
except ClickHouseError as e:
309
print(f"Failed to connect: {e}")
310
```
311
312
### Query Error Recovery
313
314
```python
315
import clickhouse_connect
316
from clickhouse_connect.driver.exceptions import (
317
ProgrammingError,
318
OperationalError,
319
DataError
320
)
321
322
client = clickhouse_connect.create_client(host='localhost')
323
324
def safe_query(client, query, parameters=None, max_retries=2):
325
"""Execute query with error handling and retry logic."""
326
327
for attempt in range(max_retries + 1):
328
try:
329
return client.query(query, parameters=parameters)
330
331
except ProgrammingError as e:
332
# Don't retry syntax errors
333
raise Exception(f"SQL syntax error: {e}")
334
335
except DataError as e:
336
# Don't retry data format errors
337
raise Exception(f"Data processing error: {e}")
338
339
except OperationalError as e:
340
if attempt < max_retries:
341
print(f"Query attempt {attempt + 1} failed: {e}")
342
print("Retrying...")
343
time.sleep(1)
344
continue
345
else:
346
raise Exception(f"Query failed after {max_retries + 1} attempts: {e}")
347
348
# Usage examples
349
try:
350
# This should work
351
result = safe_query(client, "SELECT count() FROM system.tables")
352
print(f"Table count: {result.first_item()}")
353
354
except Exception as e:
355
print(f"Query error: {e}")
356
357
try:
358
# This will fail with ProgrammingError (no retry)
359
result = safe_query(client, "SELECT * FROM invalid_syntax WHERE")
360
361
except Exception as e:
362
print(f"Syntax error (no retry): {e}")
363
364
client.close()
365
```
366
367
### Streaming Error Handling
368
369
```python
370
import clickhouse_connect
371
from clickhouse_connect.driver.exceptions import (
372
StreamClosedError,
373
StreamFailureError,
374
OperationalError
375
)
376
377
client = clickhouse_connect.create_client(host='localhost')
378
379
def process_large_dataset(client, query):
380
"""Process large dataset with streaming and error recovery."""
381
382
processed_rows = 0
383
failed_batches = 0
384
385
try:
386
for batch in client.query_row_block_stream(
387
query,
388
settings={'max_block_size': 10000}
389
):
390
try:
391
# Process batch
392
process_batch(batch)
393
processed_rows += len(batch)
394
395
except Exception as e:
396
print(f"Error processing batch: {e}")
397
failed_batches += 1
398
continue # Continue with next batch
399
400
except StreamClosedError as e:
401
print(f"Stream was closed unexpectedly: {e}")
402
print(f"Processed {processed_rows} rows before stream closed")
403
404
except StreamFailureError as e:
405
print(f"Stream failed: {e}")
406
print(f"Processed {processed_rows} rows before failure")
407
408
except OperationalError as e:
409
print(f"Network or server error during streaming: {e}")
410
print(f"Processed {processed_rows} rows before error")
411
412
finally:
413
print(f"Streaming completed. Processed: {processed_rows}, Failed batches: {failed_batches}")
414
415
def process_batch(batch):
416
"""Process a single batch of data."""
417
# Simulate processing that might fail
418
for row in batch:
419
if len(row) == 0: # Example error condition
420
raise ValueError("Empty row encountered")
421
# Process row...
422
423
# Usage
424
process_large_dataset(client, "SELECT * FROM large_table")
425
client.close()
426
```
427
428
### DB-API Exception Compatibility
429
430
```python
431
import clickhouse_connect.dbapi
432
from clickhouse_connect.driver.exceptions import (
433
Error,
434
DatabaseError,
435
OperationalError,
436
ProgrammingError
437
)
438
439
# DB-API compatible error handling
440
conn = clickhouse_connect.dbapi.connect(host='localhost')
441
cursor = conn.cursor()
442
443
def db_api_operations():
444
"""Example of DB-API error handling patterns."""
445
446
try:
447
# Execute potentially problematic query
448
cursor.execute("SELECT * FROM users WHERE id = %s", (999,))
449
result = cursor.fetchall()
450
451
if not result:
452
print("No results found")
453
else:
454
for row in result:
455
print(row)
456
457
except ProgrammingError as e:
458
# SQL syntax or programming errors
459
print(f"Programming error: {e}")
460
return False
461
462
except OperationalError as e:
463
# Database operational errors
464
print(f"Operational error: {e}")
465
return False
466
467
except DatabaseError as e:
468
# Any other database error
469
print(f"Database error: {e}")
470
return False
471
472
except Error as e:
473
# Catch-all for DB-API errors
474
print(f"DB-API error: {e}")
475
return False
476
477
return True
478
479
# Usage
480
success = db_api_operations()
481
if success:
482
print("Operations completed successfully")
483
else:
484
print("Operations failed")
485
486
cursor.close()
487
conn.close()
488
```
489
490
### Custom Error Handling Context
491
492
```python
493
import clickhouse_connect
494
from clickhouse_connect.driver.exceptions import ClickHouseError
495
from contextlib import contextmanager
496
import logging
497
498
# Set up logging
499
logging.basicConfig(level=logging.INFO)
500
logger = logging.getLogger(__name__)
501
502
@contextmanager
503
def clickhouse_error_handler(operation_name="database operation"):
504
"""Context manager for consistent error handling."""
505
506
try:
507
yield
508
logger.info(f"{operation_name} completed successfully")
509
510
except ProgrammingError as e:
511
logger.error(f"{operation_name} failed - Programming error: {e}")
512
raise
513
514
except OperationalError as e:
515
logger.error(f"{operation_name} failed - Operational error: {e}")
516
raise
517
518
except DataError as e:
519
logger.error(f"{operation_name} failed - Data error: {e}")
520
raise
521
522
except ClickHouseError as e:
523
logger.error(f"{operation_name} failed - ClickHouse error: {e}")
524
raise
525
526
except Exception as e:
527
logger.error(f"{operation_name} failed - Unexpected error: {e}")
528
raise
529
530
# Usage with context manager
531
client = clickhouse_connect.create_client(host='localhost')
532
533
with clickhouse_error_handler("User data query"):
534
result = client.query("SELECT * FROM users LIMIT 10")
535
print(f"Retrieved {len(result.result_set)} users")
536
537
with clickhouse_error_handler("User data insertion"):
538
client.insert('users', [
539
['John', 25, 'Engineer'],
540
['Jane', 30, 'Manager']
541
], column_names=['name', 'age', 'role'])
542
543
client.close()
544
```