CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-clickhouse-connect

ClickHouse Database Core Driver for Python, Pandas, and Superset

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Overview
Eval results
Files

exceptions.mddocs/

Exception Handling

Comprehensive exception hierarchy providing detailed error information and compatibility with DB-API 2.0 exception model for robust error handling in ClickHouse applications.

Capabilities

Base Exception Classes

Foundation exception classes providing the core error handling framework for all ClickHouse Connect operations.

class ClickHouseError(Exception):
    """
    Base exception class for all ClickHouse-related errors.
    
    All ClickHouse Connect exceptions inherit from this base class,
    allowing for broad exception handling patterns.
    """
    pass

class Error(Exception):
    """
    Base class for all non-warning database errors.
    
    Compatible with DB-API 2.0 exception hierarchy.
    Serves as the root for all error conditions that
    should halt program execution.
    """
    pass

class Warning(Exception):
    """
    Exception for important warnings.
    
    Used for non-fatal issues that should be brought
    to the attention of the application developer.
    """
    pass

Database Error Categories

DB-API 2.0 compatible exception hierarchy providing standardized error categorization for different types of database-related failures.

class InterfaceError(Error):
    """
    Exception for database interface errors.
    
    Raised when there are problems with the database interface
    itself rather than the database. Examples include:
    - Connection configuration errors
    - Protocol version mismatches
    - Client library issues
    """
    pass

class DatabaseError(Error):
    """
    Base exception for database-related errors.
    
    Parent class for all exceptions that are related to
    the database operation itself. All other database
    exception types inherit from this class.
    """
    pass

class DataError(DatabaseError):
    """
    Exception for data processing errors.
    
    Raised when there are problems processing data values:
    - Type conversion failures
    - Invalid data formats
    - Data truncation issues
    - Encoding/decoding problems
    """
    pass

class OperationalError(DatabaseError):
    """
    Exception for database operation errors.
    
    Raised for errors related to database operations
    rather than application programming errors:
    - Connection failures
    - Network timeouts
    - Server unavailable
    - Transaction conflicts
    - Memory/resource limits exceeded
    """
    pass

class IntegrityError(DatabaseError):
    """
    Exception for referential integrity errors.
    
    Raised when database relational integrity is affected:
    - Constraint violations
    - Duplicate key errors
    - Foreign key constraint failures
    
    Note: ClickHouse has limited constraint support,
    so this is mainly for compatibility.
    """
    pass

class InternalError(DatabaseError):
    """
    Exception for database internal errors.
    
    Raised when the database encounters internal errors:
    - Internal database server errors
    - Corruption issues
    - System resource failures
    """
    pass

class ProgrammingError(DatabaseError):
    """
    Exception for programming errors.
    
    Raised for errors in the application's use of the database:
    - SQL syntax errors
    - Invalid table/column names
    - Wrong number of parameters
    - Incorrect parameter types
    - Invalid operation for current state
    """
    pass

class NotSupportedError(DatabaseError):
    """
    Exception for unsupported operations.
    
    Raised when attempting to use features not supported
    by ClickHouse or the current configuration:
    - Unsupported SQL features
    - Unsupported data types
    - Operations not available in current version
    """
    pass

Streaming-Specific Exceptions

Specialized exceptions for handling errors in streaming operations and data transfer scenarios.

class StreamClosedError(ClickHouseError):
    """
    Exception for operations on closed streams.
    
    Raised when attempting to perform operations on
    a stream that has already been closed or finalized:
    - Reading from closed result stream
    - Writing to closed insert stream
    - Accessing stream after connection closed
    """
    pass

class StreamFailureError(ClickHouseError):
    """
    Exception for unexpected stream failures.
    
    Raised when stream operations fail due to
    unexpected conditions:
    - Network interruption during streaming
    - Server-side stream termination
    - Buffer overflow conditions
    - Compression/decompression failures
    """
    pass

Connection and Authentication Errors

Exceptions specifically related to connection establishment, authentication, and network-level communication issues.

class ConnectionError(OperationalError):
    """
    Exception for connection-related errors.
    
    Raised when connection to ClickHouse server fails:
    - Cannot connect to host/port
    - Connection refused
    - DNS resolution failures
    - SSL/TLS handshake failures
    """
    pass

class AuthenticationError(OperationalError):
    """
    Exception for authentication failures.
    
    Raised when authentication with ClickHouse server fails:
    - Invalid username/password
    - Expired access tokens
    - Insufficient permissions
    - Authentication method not supported
    """
    pass

class TimeoutError(OperationalError):
    """
    Exception for timeout conditions.
    
    Raised when operations exceed configured timeout limits:
    - Connection timeout
    - Query execution timeout
    - Network read/write timeout
    """
    pass

Usage Examples

Basic Exception Handling

import clickhouse_connect
from clickhouse_connect.driver.exceptions import (
    ClickHouseError,
    ProgrammingError,
    OperationalError,
    DataError
)

client = clickhouse_connect.create_client(host='localhost')

try:
    # This might raise various exceptions
    result = client.query("SELECT * FROM non_existent_table")
    
except ProgrammingError as e:
    print(f"SQL error: {e}")
    # Handle syntax errors, invalid table names, etc.
    
except OperationalError as e:
    print(f"Operation failed: {e}")
    # Handle connection issues, timeouts, server errors
    
except DataError as e:
    print(f"Data processing error: {e}")
    # Handle type conversion, format issues
    
except ClickHouseError as e:
    print(f"ClickHouse error: {e}")
    # Catch any other ClickHouse-specific errors
    
except Exception as e:
    print(f"Unexpected error: {e}")
    # Handle any other unexpected errors
    
finally:
    client.close()

Connection Error Handling

import clickhouse_connect
from clickhouse_connect.driver.exceptions import (
    ConnectionError,
    AuthenticationError,
    TimeoutError,
    OperationalError
)

def create_robust_client(host, max_retries=3):
    """Create client with retry logic for connection issues."""
    
    for attempt in range(max_retries):
        try:
            client = clickhouse_connect.create_client(
                host=host,
                connect_timeout=10,
                send_receive_timeout=60
            )
            
            # Test connection
            client.ping()
            return client
            
        except ConnectionError as e:
            print(f"Connection attempt {attempt + 1} failed: {e}")
            if attempt == max_retries - 1:
                raise
            time.sleep(2 ** attempt)  # Exponential backoff
            
        except AuthenticationError as e:
            print(f"Authentication failed: {e}")
            # Don't retry authentication errors
            raise
            
        except TimeoutError as e:
            print(f"Connection timeout: {e}")
            if attempt == max_retries - 1:
                raise
            time.sleep(1)
            
    raise OperationalError("Failed to establish connection after all retries")

# Usage
try:
    client = create_robust_client('clickhouse.example.com')
    print("Connection established successfully")
except ClickHouseError as e:
    print(f"Failed to connect: {e}")

Query Error Recovery

import clickhouse_connect
from clickhouse_connect.driver.exceptions import (
    ProgrammingError,
    OperationalError,
    DataError
)

client = clickhouse_connect.create_client(host='localhost')

def safe_query(client, query, parameters=None, max_retries=2):
    """Execute query with error handling and retry logic."""
    
    for attempt in range(max_retries + 1):
        try:
            return client.query(query, parameters=parameters)
            
        except ProgrammingError as e:
            # Don't retry syntax errors
            raise Exception(f"SQL syntax error: {e}")
            
        except DataError as e:
            # Don't retry data format errors
            raise Exception(f"Data processing error: {e}")
            
        except OperationalError as e:
            if attempt < max_retries:
                print(f"Query attempt {attempt + 1} failed: {e}")
                print("Retrying...")
                time.sleep(1)
                continue
            else:
                raise Exception(f"Query failed after {max_retries + 1} attempts: {e}")

# Usage examples
try:
    # This should work
    result = safe_query(client, "SELECT count() FROM system.tables")
    print(f"Table count: {result.first_item()}")
    
except Exception as e:
    print(f"Query error: {e}")

try:
    # This will fail with ProgrammingError (no retry)
    result = safe_query(client, "SELECT * FROM invalid_syntax WHERE")
    
except Exception as e:
    print(f"Syntax error (no retry): {e}")

client.close()

Streaming Error Handling

import clickhouse_connect
from clickhouse_connect.driver.exceptions import (
    StreamClosedError,
    StreamFailureError,
    OperationalError
)

client = clickhouse_connect.create_client(host='localhost')

def process_large_dataset(client, query):
    """Process large dataset with streaming and error recovery."""
    
    processed_rows = 0
    failed_batches = 0
    
    try:
        for batch in client.query_row_block_stream(
            query,
            settings={'max_block_size': 10000}
        ):
            try:
                # Process batch
                process_batch(batch)
                processed_rows += len(batch)
                
            except Exception as e:
                print(f"Error processing batch: {e}")
                failed_batches += 1
                continue  # Continue with next batch
                
    except StreamClosedError as e:
        print(f"Stream was closed unexpectedly: {e}")
        print(f"Processed {processed_rows} rows before stream closed")
        
    except StreamFailureError as e:
        print(f"Stream failed: {e}")
        print(f"Processed {processed_rows} rows before failure")
        
    except OperationalError as e:
        print(f"Network or server error during streaming: {e}")
        print(f"Processed {processed_rows} rows before error")
        
    finally:
        print(f"Streaming completed. Processed: {processed_rows}, Failed batches: {failed_batches}")

def process_batch(batch):
    """Process a single batch of data."""
    # Simulate processing that might fail
    for row in batch:
        if len(row) == 0:  # Example error condition
            raise ValueError("Empty row encountered")
        # Process row...

# Usage
process_large_dataset(client, "SELECT * FROM large_table")
client.close()

DB-API Exception Compatibility

import clickhouse_connect.dbapi
from clickhouse_connect.driver.exceptions import (
    Error,
    DatabaseError,
    OperationalError,
    ProgrammingError
)

# DB-API compatible error handling
conn = clickhouse_connect.dbapi.connect(host='localhost')
cursor = conn.cursor()

def db_api_operations():
    """Example of DB-API error handling patterns."""
    
    try:
        # Execute potentially problematic query
        cursor.execute("SELECT * FROM users WHERE id = %s", (999,))
        result = cursor.fetchall()
        
        if not result:
            print("No results found")
        else:
            for row in result:
                print(row)
                
    except ProgrammingError as e:
        # SQL syntax or programming errors
        print(f"Programming error: {e}")
        return False
        
    except OperationalError as e:
        # Database operational errors
        print(f"Operational error: {e}")
        return False
        
    except DatabaseError as e:
        # Any other database error
        print(f"Database error: {e}")
        return False
        
    except Error as e:
        # Catch-all for DB-API errors
        print(f"DB-API error: {e}")
        return False
        
    return True

# Usage
success = db_api_operations()
if success:
    print("Operations completed successfully")
else:
    print("Operations failed")

cursor.close()
conn.close()

Custom Error Handling Context

import clickhouse_connect
from clickhouse_connect.driver.exceptions import ClickHouseError
from contextlib import contextmanager
import logging

# Set up logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

@contextmanager  
def clickhouse_error_handler(operation_name="database operation"):
    """Context manager for consistent error handling."""
    
    try:
        yield
        logger.info(f"{operation_name} completed successfully")
        
    except ProgrammingError as e:
        logger.error(f"{operation_name} failed - Programming error: {e}")
        raise
        
    except OperationalError as e:
        logger.error(f"{operation_name} failed - Operational error: {e}")
        raise
        
    except DataError as e:
        logger.error(f"{operation_name} failed - Data error: {e}")
        raise
        
    except ClickHouseError as e:
        logger.error(f"{operation_name} failed - ClickHouse error: {e}")
        raise
        
    except Exception as e:
        logger.error(f"{operation_name} failed - Unexpected error: {e}")
        raise

# Usage with context manager
client = clickhouse_connect.create_client(host='localhost')

with clickhouse_error_handler("User data query"):
    result = client.query("SELECT * FROM users LIMIT 10")
    print(f"Retrieved {len(result.result_set)} users")

with clickhouse_error_handler("User data insertion"):
    client.insert('users', [
        ['John', 25, 'Engineer'],
        ['Jane', 30, 'Manager']
    ], column_names=['name', 'age', 'role'])

client.close()

Install with Tessl CLI

npx tessl i tessl/pypi-clickhouse-connect

docs

client-api.md

data-formats.md

dbapi.md

exceptions.md

index.md

sqlalchemy.md

utilities.md

tile.json