0
# Table Construction and Data I/O
1
2
Core functions for creating table expressions from various data sources including in-memory data, files, and database connections.
3
4
## Capabilities
5
6
### Table Schema Definition
7
8
Create table expressions from schema definitions for building queries before data materialization.
9
10
```python { .api }
11
def table(schema=None, name=None, catalog=None, database=None):
12
"""
13
Create a table expression from a schema.
14
15
Parameters:
16
- schema: dict, Schema, or list of (name, type) tuples defining table structure (optional)
17
- name: str, optional name for the table
18
- catalog: str, optional catalog name
19
- database: str, optional database name
20
21
Returns:
22
Table: Table expression
23
"""
24
```
25
26
**Usage Example:**
27
```python
28
import ibis
29
30
# From dictionary
31
schema = {'name': 'string', 'age': 'int64', 'salary': 'float64'}
32
t = ibis.table(schema, name='employees')
33
34
# From Schema object
35
schema = ibis.schema([('name', 'string'), ('age', 'int64')])
36
t = ibis.table(schema)
37
```
38
39
### In-Memory Tables
40
41
Create table expressions from in-memory data structures like pandas DataFrames, dictionaries, or lists.
42
43
```python { .api }
44
def memtable(data, /, *, columns=None, schema=None, name=None):
45
"""
46
Create a table expression from in-memory data.
47
48
Parameters:
49
- data: pandas.DataFrame, dict, list of dicts, or pyarrow.Table (positional-only)
50
- columns: Iterable[str], optional column names (keyword-only)
51
- schema: Schema or schema-like, optional schema specification (keyword-only)
52
- name: str, optional name for the table (keyword-only)
53
54
Returns:
55
Table: Table expression backed by in-memory data
56
"""
57
```
58
59
**Usage Example:**
60
```python
61
import pandas as pd
62
import ibis
63
64
# From pandas DataFrame
65
df = pd.DataFrame({'x': [1, 2, 3], 'y': ['a', 'b', 'c']})
66
t = ibis.memtable(df)
67
68
# From dictionary
69
data = {'x': [1, 2, 3], 'y': ['a', 'b', 'c']}
70
t = ibis.memtable(data, name='my_table')
71
```
72
73
### CSV File Reading
74
75
Read CSV files into table expressions with configurable parsing options.
76
77
```python { .api }
78
def read_csv(paths, /, *, table_name=None, **kwargs):
79
"""
80
Read a CSV file or set of CSV files into a table expression.
81
82
Parameters:
83
- paths: str, Path, or sequence of str/Path (positional-only), file path(s) or URL(s)
84
- table_name: str, optional name for resulting table (keyword-only)
85
- **kwargs: additional backend-specific options
86
87
Returns:
88
Table expression
89
"""
90
```
91
92
**Usage Example:**
93
```python
94
import ibis
95
96
# Basic CSV reading
97
t = ibis.read_csv('data.csv')
98
99
# With options
100
t = ibis.read_csv('data.csv', table_name='sales_data')
101
```
102
103
### Parquet File Reading
104
105
Read Parquet files into table expressions with support for partitioned datasets.
106
107
```python { .api }
108
def read_parquet(paths, /, *, table_name=None, **kwargs):
109
"""
110
Read a Parquet file or dataset into a table expression.
111
112
Parameters:
113
- paths: str, Path, or sequence of str/Path (positional-only), file path(s), directory, or URL(s)
114
- table_name: str, optional name for resulting table (keyword-only)
115
- **kwargs: additional backend-specific options
116
117
Returns:
118
Table expression
119
"""
120
```
121
122
**Usage Example:**
123
```python
124
import ibis
125
126
# Single file
127
t = ibis.read_parquet('data.parquet')
128
129
# Partitioned dataset
130
t = ibis.read_parquet('partitioned_data/')
131
```
132
133
### JSON File Reading
134
135
Read JSON files into table expressions with flexible schema inference.
136
137
```python { .api }
138
def read_json(paths, /, *, table_name=None, **kwargs):
139
"""
140
Read a JSON file into a table expression.
141
142
Parameters:
143
- paths: str, Path, or sequence of str/Path (positional-only), file path(s) or URL(s)
144
- table_name: str, optional name for resulting table (keyword-only)
145
- **kwargs: additional backend-specific options
146
147
Returns:
148
Table expression
149
"""
150
```
151
152
**Usage Example:**
153
```python
154
import ibis
155
156
# Read JSON file
157
t = ibis.read_json('data.json')
158
159
# JSONL (newline-delimited JSON)
160
t = ibis.read_json('data.jsonl')
161
```
162
163
### Delta Lake Reading
164
165
Read Delta Lake tables into table expressions.
166
167
```python { .api }
168
def read_delta(path, /, *, table_name=None, **kwargs):
169
"""
170
Read a Delta Lake table into a table expression.
171
172
Parameters:
173
- path: str or Path (positional-only), path to Delta table
174
- table_name: str, optional name for resulting table (keyword-only)
175
- **kwargs: additional backend-specific options
176
177
Returns:
178
Table expression
179
"""
180
```
181
182
### Database Connections
183
184
Connect to various backends and access existing tables.
185
186
```python { .api }
187
def connect(backend=None, **kwargs):
188
"""
189
Connect to a backend.
190
191
Parameters:
192
- backend: str, backend name (duckdb, postgres, etc.)
193
- **kwargs: backend-specific connection parameters
194
195
Returns:
196
Backend connection object
197
"""
198
```
199
200
**Usage Examples:**
201
```python
202
import ibis
203
204
# DuckDB (default, in-memory)
205
con = ibis.connect()
206
207
# DuckDB with file
208
con = ibis.duckdb.connect('my_database.db')
209
210
# PostgreSQL
211
con = ibis.postgres.connect(
212
user='user',
213
password='password',
214
host='localhost',
215
database='mydb'
216
)
217
218
# BigQuery
219
con = ibis.bigquery.connect(project_id='my-project')
220
221
# Access existing table
222
table = con.table('existing_table')
223
```
224
225
## Schema Operations
226
227
### Schema Definition
228
229
```python { .api }
230
def schema(mapping_or_list):
231
"""
232
Create a schema from a mapping or list of field definitions.
233
234
Parameters:
235
- mapping_or_list: dict or list of (name, type) tuples
236
237
Returns:
238
Schema object
239
"""
240
```
241
242
### Schema Inference
243
244
```python { .api }
245
def infer_schema(data):
246
"""
247
Infer schema from data.
248
249
Parameters:
250
- data: pandas.DataFrame, dict, or other data structure
251
252
Returns:
253
Schema object with inferred types
254
"""
255
```
256
257
**Usage Example:**
258
```python
259
import pandas as pd
260
import ibis
261
262
df = pd.DataFrame({'x': [1, 2, 3], 'y': ['a', 'b', 'c']})
263
schema = ibis.infer_schema(df)
264
print(schema) # Schema({'x': int64, 'y': string})
265
```