0
# Backend Management
1

2
Backend connection, configuration, and management functions for working with different data processing engines and databases.
3

4
## Capabilities
5

6
### Backend Discovery
7

8
Get information about available and current backends.
9

10
```python { .api }
11
def get_backend(table_or_expr):
12
    """
13
    Get the backend for a table or expression.
14
    
15
    Parameters:
16
    - table_or_expr: Table or expression to get backend for
17
    
18
    Returns:
19
    Backend instance
20
    """
21

22
def set_backend(backend):
23
    """
24
    Set the default backend for new operations.
25
    
26
    Parameters:
27
    - backend: Backend instance or backend name string
28
    
29
    Returns:
30
    None
31
    """
32

33
def list_backends():
34
    """
35
    List all available backend names.
36
    
37
    Returns:
38
    List of backend name strings
39
    """
40
```
41

42
**Usage Examples:**
43
```python
44
import ibis
45

46
# Get backend for a table
47
table = ibis.memtable({'x': [1, 2, 3]})
48
backend = ibis.get_backend(table)
49
print(backend.name)  # 'duckdb'
50

51
# Set default backend
52
ibis.set_backend('duckdb')
53

54
# List available backends
55
backends = ibis.list_backends()
56
print(backends)  # ['duckdb', 'postgres', 'sqlite', ...]
57
```
58

59
### Database Backends
60

61
Connect to traditional SQL databases.
62

63
#### PostgreSQL
64

65
```python { .api }
66
ibis.postgres.connect(
67
    user=None,
68
    password=None, 
69
    host='localhost',
70
    port=5432,
71
    database=None,
72
    **kwargs
73
):
74
    """
75
    Connect to PostgreSQL database.
76
    
77
    Parameters:
78
    - user: str, username
79
    - password: str, password
80
    - host: str, hostname
81
    - port: int, port number
82
    - database: str, database name
83
    - **kwargs: additional connection parameters
84
    
85
    Returns:
86
    PostgreSQL backend connection
87
    """
88
```
89

90
#### MySQL
91

92
```python { .api }
93
ibis.mysql.connect(
94
    user=None,
95
    password=None,
96
    host='localhost', 
97
    port=3306,
98
    database=None,
99
    **kwargs
100
):
101
    """Connect to MySQL database."""
102
```
103

104
#### SQLite
105

106
```python { .api }
107
ibis.sqlite.connect(path=':memory:', **kwargs):
108
    """
109
    Connect to SQLite database.
110
    
111
    Parameters:
112
    - path: str, path to database file or ':memory:' for in-memory
113
    - **kwargs: additional connection parameters
114
    
115
    Returns:
116
    SQLite backend connection
117
    """
118
```
119

120
### Cloud Data Warehouses
121

122
Connect to cloud-based data processing systems.
123

124
#### BigQuery
125

126
```python { .api }
127
ibis.bigquery.connect(
128
    project_id=None,
129
    dataset_id=None,
130
    credentials=None,
131
    **kwargs
132
):
133
    """
134
    Connect to Google BigQuery.
135
    
136
    Parameters:
137
    - project_id: str, Google Cloud project ID
138
    - dataset_id: str, default dataset ID
139
    - credentials: authentication credentials
140
    - **kwargs: additional connection parameters
141
    
142
    Returns:
143
    BigQuery backend connection
144
    """
145
```
146

147
#### Snowflake
148

149
```python { .api }
150
ibis.snowflake.connect(
151
    user=None,
152
    password=None,
153
    account=None,
154
    database=None,
155
    warehouse=None,
156
    **kwargs
157
):
158
    """
159
    Connect to Snowflake data warehouse.
160
    
161
    Parameters:
162
    - user: str, username
163
    - password: str, password  
164
    - account: str, Snowflake account identifier
165
    - database: str, database name
166
    - warehouse: str, warehouse name
167
    - **kwargs: additional connection parameters
168
    
169
    Returns:
170
    Snowflake backend connection
171
    """
172
```
173

174
### Analytical Engines
175

176
Connect to specialized analytical and OLAP systems.
177

178
#### DuckDB
179

180
```python { .api }
181
ibis.duckdb.connect(path=':memory:', **kwargs):
182
    """
183
    Connect to DuckDB database.
184
    
185
    Parameters:
186
    - path: str, path to database file or ':memory:' for in-memory
187
    - **kwargs: additional connection parameters
188
    
189
    Returns:
190
    DuckDB backend connection
191
    """
192
```
193

194
#### ClickHouse
195

196
```python { .api }
197
ibis.clickhouse.connect(
198
    host='localhost',
199
    port=9000,
200
    user='default',
201
    password='',
202
    database='default',
203
    **kwargs
204
):
205
    """Connect to ClickHouse database."""
206
```
207

208
### Distributed Systems
209

210
Connect to distributed computing frameworks.
211

212
#### Apache Spark
213

214
```python { .api }
215
ibis.spark.connect(session=None, **kwargs):
216
    """
217
    Connect to Apache Spark.
218
    
219
    Parameters:
220
    - session: existing SparkSession instance
221
    - **kwargs: SparkSession configuration parameters
222
    
223
    Returns:
224
    Spark backend connection
225
    """
226
```
227

228
#### Trino
229

230
```python { .api }
231
ibis.trino.connect(
232
    host='localhost',
233
    port=8080,
234
    user=None,
235
    catalog=None,
236
    schema=None,
237
    **kwargs
238
):
239
    """Connect to Trino distributed query engine."""
240
```
241

242
### In-Memory Engines
243

244
Connect to in-memory data processing systems.
245

246
#### Polars
247

248
```python { .api }
249
ibis.polars.connect(**kwargs):
250
    """
251
    Connect to Polars backend.
252
    
253
    Returns:
254
    Polars backend connection
255
    """
256
```
257

258
#### Pandas
259

260
```python { .api }
261
ibis.pandas.connect(**kwargs):
262
    """
263
    Connect to Pandas backend.
264
    
265
    Returns:
266
    Pandas backend connection
267
    """
268
```
269

270
#### Dask
271

272
```python { .api }
273
ibis.dask.connect(client=None, **kwargs):
274
    """
275
    Connect to Dask backend.
276
    
277
    Parameters:
278
    - client: Dask client instance
279
    - **kwargs: additional parameters
280
    
281
    Returns:
282
    Dask backend connection
283
    """
284
```
285

286
### Backend Operations
287

288
Common operations available on all backend connections.
289

290
```python { .api }
291
# Backend connection methods
292
backend.table(name): Table
293
    """Get existing table by name."""
294

295
backend.create_table(name, data): Table
296
    """Create table from data."""
297

298
backend.list_tables(): List[str]  
299
    """List available table names."""
300

301
backend.compile(expr): str
302
    """Compile expression to backend-specific query."""
303

304
backend.execute(expr): Any
305
    """Execute expression and return results."""
306

307
backend.has_operation(operation): bool
308
    """Check if backend supports operation."""
309
```
310

311
**Usage Examples:**
312
```python
313
# Connect to different backends
314
duckdb_con = ibis.duckdb.connect('my_db.duckdb')
315
pg_con = ibis.postgres.connect(
316
    user='user', 
317
    password='pass',
318
    database='mydb'
319
)
320

321
# Common operations
322
tables = duckdb_con.list_tables()
323
my_table = duckdb_con.table('existing_table')
324

325
# Create table
326
import pandas as pd
327
df = pd.DataFrame({'x': [1, 2, 3], 'y': ['a', 'b', 'c']})
328
new_table = duckdb_con.create_table('new_data', df)
329

330
# Compile and execute
331
expr = my_table.filter(my_table.x > 1)
332
sql = duckdb_con.compile(expr)
333
result = duckdb_con.execute(expr)
334
```
335

336
### Cross-Backend Operations
337

338
Move data and expressions between backends.
339

340
**Usage Examples:**
341
```python
342
# Create expression on one backend
343
duckdb_con = ibis.duckdb.connect()
344
pg_con = ibis.postgres.connect(...)
345

346
duckdb_table = duckdb_con.table('source_data')
347
expr = duckdb_table.filter(duckdb_table.value > 100)
348

349
# Execute same expression on different backend
350
# (assuming compatible table exists)
351
pg_table = pg_con.table('source_data')
352
same_expr = pg_table.filter(pg_table.value > 100)
353
pg_result = pg_con.execute(same_expr)
354

355
# Transfer data between backends
356
data = duckdb_con.execute(expr).to_pandas()
357
pg_table = pg_con.create_table('transferred_data', data)
358
```
359

360
### Backend-Specific Features
361

362
Access backend-specific functionality.
363

364
```python { .api }
365
# Backend-specific methods (examples)
366
bigquery_backend.create_dataset(name): None
367
spark_backend.spark_session: SparkSession
368
postgres_backend.raw_sql(query): ResultProxy
369
```
370

371
**Usage Examples:**
372
```python
373
# BigQuery specific operations
374
bq_con = ibis.bigquery.connect(project_id='my-project')
375
bq_con.create_dataset('analytics')
376

377
# Spark specific operations  
378
spark_con = ibis.spark.connect()
379
spark_session = spark_con.spark_session
380

381
# Raw SQL execution
382
pg_con = ibis.postgres.connect(...)
383
result = pg_con.raw_sql('SELECT version()')
384
```
385

386
### URL-Based Connections
387

388
Connect using database URLs.
389

390
```python { .api }
391
backend._from_url(url):
392
    """
393
    Connect using database URL.
394
    
395
    Parameters:
396
    - url: str, database connection URL
397
    
398
    Returns:
399
    Backend connection
400
    """
401
```
402

403
**Usage Examples:**
404
```python
405
# URL-based connections
406
pg_con = ibis.postgres._from_url('postgresql://user:pass@localhost/mydb')
407
sqlite_con = ibis.sqlite._from_url('sqlite:///path/to/database.db')
408
```