tessl/pypi-ibis-framework

The portable Python dataframe library that provides a unified API for data analysis across 20+ different backends

—

Pending

Overview

Eval results

Files

Configuration

Name: tessl/pypi-ibis-framework
Author: tessl

Global and backend-specific configuration system for controlling behavior, output formatting, and performance optimizations.

Capabilities

Configuration Access

Access and modify Ibis configuration options.

ibis.options: OptionsManager
    """Global options manager for Ibis configuration."""

ibis.options.get(key): Any
    """Get configuration value."""

ibis.options.set(key, value): None
    """Set configuration value."""

ibis.options.reset(): None
    """Reset all options to defaults."""

Usage Examples:

import ibis

# Access options
print(ibis.options.sql.default_limit)

# Set options
ibis.options.sql.default_limit = 1000
ibis.options.interactive.mode = True

# Reset to defaults
ibis.options.reset()

SQL Configuration

Control SQL compilation and execution behavior.

ibis.options.sql.default_limit: int
    """Default row limit for query results (default: 10000)."""

ibis.options.sql.default_backend: str
    """Default backend name for new operations."""

Usage Examples:

# Set default row limit
ibis.options.sql.default_limit = 5000

# All queries will be limited to 5000 rows by default
result = table.select('*')  # Automatically adds LIMIT 5000

# Override for specific query
unlimited = table.select('*').limit(None)  # No limit

# Set default backend
ibis.options.sql.default_backend = 'duckdb'

Interactive Mode Configuration

Control behavior in interactive environments like Jupyter.

ibis.options.interactive.mode: bool
    """Enable interactive mode (default: True in Jupyter, False otherwise)."""

ibis.options.interactive.max_rows: int
    """Maximum rows to display in interactive repr (default: 10)."""

ibis.options.interactive.max_columns: int  
    """Maximum columns to display in interactive repr (default: None)."""

ibis.options.interactive.max_length: int
    """Maximum string length to display (default: 100)."""

ibis.options.interactive.max_depth: int
    """Maximum nesting depth for nested data (default: 1)."""

Usage Examples:

# Configure interactive display
ibis.options.interactive.mode = True
ibis.options.interactive.max_rows = 20
ibis.options.interactive.max_columns = 8
ibis.options.interactive.max_length = 50

# Now table expressions will display nicely in Jupyter
table  # Shows formatted preview with configured limits

Representation Configuration

Control how expressions and results are displayed.

ibis.options.repr.interactive.show_types: bool
    """Show column types in table repr (default: True)."""

ibis.options.repr.interactive.table_columns: int
    """Number of table columns to show (default: None for all)."""

ibis.options.repr.query_text_length: int
    """Maximum query text length in repr (default: 200)."""

Usage Examples:

# Configure table representation
ibis.options.repr.interactive.show_types = False  # Hide column types
ibis.options.repr.interactive.table_columns = 5   # Show max 5 columns
ibis.options.repr.query_text_length = 100        # Truncate long queries

# Display configuration affects how tables look
print(table)  # Uses configured representation settings

Backend-Specific Configuration

Configure behavior for specific backends.

Usage Examples:

# DuckDB-specific options
ibis.options.duckdb.temp_directory = '/tmp/duckdb'
ibis.options.duckdb.memory_limit = '4GB'

# BigQuery-specific options  
ibis.options.bigquery.project_id = 'my-default-project'
ibis.options.bigquery.dataset_id = 'analytics'

# PostgreSQL connection pooling
ibis.options.postgres.pool_size = 10
ibis.options.postgres.max_overflow = 20

Context Managers

Temporarily change configuration using context managers.

ibis.options.with_config(**kwargs): ContextManager
    """Temporarily set configuration options."""

Usage Examples:

# Temporarily change settings
with ibis.options.with_config(sql_default_limit=100):
    # All queries in this block use limit of 100
    result1 = table.select('*')
    result2 = other_table.select('*')

# Outside the block, original limit is restored
result3 = table.select('*')  # Uses original default limit

# Multiple options at once
with ibis.options.with_config(
    interactive_mode=False,
    sql_default_limit=50,
    repr_show_types=False
):
    # Custom behavior for this block
    print(table)

Configuration Validation

Validate configuration values.

Usage Examples:

# Invalid configuration raises errors
try:
    ibis.options.sql.default_limit = -1  # Invalid negative limit
except ValueError as e:
    print(f"Configuration error: {e}")

try:
    ibis.options.interactive.max_rows = "invalid"  # Wrong type
except TypeError as e:
    print(f"Type error: {e}")

Configuration Persistence

Save and load configuration settings.

Usage Examples:

# Get current configuration as dict
current_config = ibis.options.to_dict()

# Save to file (conceptual example)
import json
with open('ibis_config.json', 'w') as f:
    json.dump(current_config, f)

# Load configuration (conceptual example)
with open('ibis_config.json', 'r') as f:
    saved_config = json.load(f)

# Apply saved configuration
for key, value in saved_config.items():
    ibis.options.set(key, value)

Performance Configuration

Options that affect query execution performance.

ibis.options.compute.default_cache: bool
    """Enable result caching by default (default: False)."""

ibis.options.compute.chunk_size: int  
    """Default chunk size for streaming operations."""

ibis.options.compute.parallel: bool
    """Enable parallel execution where supported."""

Usage Examples:

# Performance tuning
ibis.options.compute.default_cache = True     # Cache results
ibis.options.compute.chunk_size = 10000      # Process in 10k row chunks
ibis.options.compute.parallel = True         # Use parallel processing

# These affect how backends execute queries
result = large_table.group_by('category').aggregate(total=large_table.value.sum())

Debugging Configuration

Options for debugging and development.

ibis.options.debug.verbose: bool
    """Enable verbose logging (default: False)."""

ibis.options.debug.show_sql: bool
    """Show generated SQL queries (default: False)."""

ibis.options.debug.show_execution_time: bool
    """Show query execution times (default: False)."""

Usage Examples:

# Enable debugging
ibis.options.debug.verbose = True
ibis.options.debug.show_sql = True
ibis.options.debug.show_execution_time = True

# Now operations show debug information
result = table.filter(table.value > 100).execute()
# Prints:
# SQL: SELECT * FROM table WHERE value > 100
# Execution time: 0.045 seconds

Option Discovery

Find available configuration options.

Usage Examples:

# List all available options
all_options = ibis.options.list_options()
print("Available options:")
for option in all_options:
    print(f"  {option}: {ibis.options.get(option)}")

# Get options by category
sql_options = ibis.options.list_options(category='sql')
interactive_options = ibis.options.list_options(category='interactive')

# Get option documentation
help_text = ibis.options.get_help('sql.default_limit')
print(help_text)

Configuration Inheritance

How configuration is inherited and overridden.

Usage Examples:

# Global configuration
ibis.options.sql.default_limit = 1000

# Backend-specific override
con = ibis.duckdb.connect()
con.options.sql.default_limit = 500  # Override for this connection

# Query-specific override
result = table.limit(100)  # Override for this specific query

# Precedence: query-specific > connection-specific > global

Environment Variable Configuration

Configure Ibis through environment variables.

Usage Examples:

# Environment variables (set before importing ibis)
import os
os.environ['IBIS_SQL_DEFAULT_LIMIT'] = '5000'
os.environ['IBIS_INTERACTIVE_MODE'] = 'true' 
os.environ['IBIS_DEBUG_VERBOSE'] = 'false'

import ibis

# Options are automatically set from environment
print(ibis.options.sql.default_limit)  # 5000
print(ibis.options.interactive.mode)   # True
print(ibis.options.debug.verbose)      # False

Configuration Profiles

Use predefined configuration profiles.

Usage Examples:

# Apply predefined profiles
ibis.options.load_profile('development')
# Sets: debug.verbose=True, debug.show_sql=True, sql.default_limit=100

ibis.options.load_profile('production')  
# Sets: debug.verbose=False, sql.default_limit=10000, compute.parallel=True

ibis.options.load_profile('jupyter')
# Sets: interactive.mode=True, interactive.max_rows=20, repr.show_types=True

# Custom profile
custom_profile = {
    'sql.default_limit': 2000,
    'interactive.max_rows': 15,
    'debug.show_sql': True
}
ibis.options.load_profile(custom_profile)

Install with Tessl CLI