or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

Files

docs

aggregation-windows.mdbackends.mdconfiguration.mdexpressions.mdindex.mdselectors.mdsql-integration.mdtable-construction.mdtable-operations.mdtemporal.mdudfs.md

table-construction.mddocs/

0

# Table Construction and Data I/O

1

2

Core functions for creating table expressions from various data sources including in-memory data, files, and database connections.

3

4

## Capabilities

5

6

### Table Schema Definition

7

8

Create table expressions from schema definitions for building queries before data materialization.

9

10

```python { .api }

11

def table(schema=None, name=None, catalog=None, database=None):

12

"""

13

Create a table expression from a schema.

14

15

Parameters:

16

- schema: dict, Schema, or list of (name, type) tuples defining table structure (optional)

17

- name: str, optional name for the table

18

- catalog: str, optional catalog name

19

- database: str, optional database name

20

21

Returns:

22

Table: Table expression

23

"""

24

```

25

26

**Usage Example:**

27

```python

28

import ibis

29

30

# From dictionary

31

schema = {'name': 'string', 'age': 'int64', 'salary': 'float64'}

32

t = ibis.table(schema, name='employees')

33

34

# From Schema object

35

schema = ibis.schema([('name', 'string'), ('age', 'int64')])

36

t = ibis.table(schema)

37

```

38

39

### In-Memory Tables

40

41

Create table expressions from in-memory data structures like pandas DataFrames, dictionaries, or lists.

42

43

```python { .api }

44

def memtable(data, /, *, columns=None, schema=None, name=None):

45

"""

46

Create a table expression from in-memory data.

47

48

Parameters:

49

- data: pandas.DataFrame, dict, list of dicts, or pyarrow.Table (positional-only)

50

- columns: Iterable[str], optional column names (keyword-only)

51

- schema: Schema or schema-like, optional schema specification (keyword-only)

52

- name: str, optional name for the table (keyword-only)

53

54

Returns:

55

Table: Table expression backed by in-memory data

56

"""

57

```

58

59

**Usage Example:**

60

```python

61

import pandas as pd

62

import ibis

63

64

# From pandas DataFrame

65

df = pd.DataFrame({'x': [1, 2, 3], 'y': ['a', 'b', 'c']})

66

t = ibis.memtable(df)

67

68

# From dictionary

69

data = {'x': [1, 2, 3], 'y': ['a', 'b', 'c']}

70

t = ibis.memtable(data, name='my_table')

71

```

72

73

### CSV File Reading

74

75

Read CSV files into table expressions with configurable parsing options.

76

77

```python { .api }

78

def read_csv(paths, /, *, table_name=None, **kwargs):

79

"""

80

Read a CSV file or set of CSV files into a table expression.

81

82

Parameters:

83

- paths: str, Path, or sequence of str/Path (positional-only), file path(s) or URL(s)

84

- table_name: str, optional name for resulting table (keyword-only)

85

- **kwargs: additional backend-specific options

86

87

Returns:

88

Table expression

89

"""

90

```

91

92

**Usage Example:**

93

```python

94

import ibis

95

96

# Basic CSV reading

97

t = ibis.read_csv('data.csv')

98

99

# With options

100

t = ibis.read_csv('data.csv', table_name='sales_data')

101

```

102

103

### Parquet File Reading

104

105

Read Parquet files into table expressions with support for partitioned datasets.

106

107

```python { .api }

108

def read_parquet(paths, /, *, table_name=None, **kwargs):

109

"""

110

Read a Parquet file or dataset into a table expression.

111

112

Parameters:

113

- paths: str, Path, or sequence of str/Path (positional-only), file path(s), directory, or URL(s)

114

- table_name: str, optional name for resulting table (keyword-only)

115

- **kwargs: additional backend-specific options

116

117

Returns:

118

Table expression

119

"""

120

```

121

122

**Usage Example:**

123

```python

124

import ibis

125

126

# Single file

127

t = ibis.read_parquet('data.parquet')

128

129

# Partitioned dataset

130

t = ibis.read_parquet('partitioned_data/')

131

```

132

133

### JSON File Reading

134

135

Read JSON files into table expressions with flexible schema inference.

136

137

```python { .api }

138

def read_json(paths, /, *, table_name=None, **kwargs):

139

"""

140

Read a JSON file into a table expression.

141

142

Parameters:

143

- paths: str, Path, or sequence of str/Path (positional-only), file path(s) or URL(s)

144

- table_name: str, optional name for resulting table (keyword-only)

145

- **kwargs: additional backend-specific options

146

147

Returns:

148

Table expression

149

"""

150

```

151

152

**Usage Example:**

153

```python

154

import ibis

155

156

# Read JSON file

157

t = ibis.read_json('data.json')

158

159

# JSONL (newline-delimited JSON)

160

t = ibis.read_json('data.jsonl')

161

```

162

163

### Delta Lake Reading

164

165

Read Delta Lake tables into table expressions.

166

167

```python { .api }

168

def read_delta(path, /, *, table_name=None, **kwargs):

169

"""

170

Read a Delta Lake table into a table expression.

171

172

Parameters:

173

- path: str or Path (positional-only), path to Delta table

174

- table_name: str, optional name for resulting table (keyword-only)

175

- **kwargs: additional backend-specific options

176

177

Returns:

178

Table expression

179

"""

180

```

181

182

### Database Connections

183

184

Connect to various backends and access existing tables.

185

186

```python { .api }

187

def connect(backend=None, **kwargs):

188

"""

189

Connect to a backend.

190

191

Parameters:

192

- backend: str, backend name (duckdb, postgres, etc.)

193

- **kwargs: backend-specific connection parameters

194

195

Returns:

196

Backend connection object

197

"""

198

```

199

200

**Usage Examples:**

201

```python

202

import ibis

203

204

# DuckDB (default, in-memory)

205

con = ibis.connect()

206

207

# DuckDB with file

208

con = ibis.duckdb.connect('my_database.db')

209

210

# PostgreSQL

211

con = ibis.postgres.connect(

212

user='user',

213

password='password',

214

host='localhost',

215

database='mydb'

216

)

217

218

# BigQuery

219

con = ibis.bigquery.connect(project_id='my-project')

220

221

# Access existing table

222

table = con.table('existing_table')

223

```

224

225

## Schema Operations

226

227

### Schema Definition

228

229

```python { .api }

230

def schema(mapping_or_list):

231

"""

232

Create a schema from a mapping or list of field definitions.

233

234

Parameters:

235

- mapping_or_list: dict or list of (name, type) tuples

236

237

Returns:

238

Schema object

239

"""

240

```

241

242

### Schema Inference

243

244

```python { .api }

245

def infer_schema(data):

246

"""

247

Infer schema from data.

248

249

Parameters:

250

- data: pandas.DataFrame, dict, or other data structure

251

252

Returns:

253

Schema object with inferred types

254

"""

255

```

256

257

**Usage Example:**

258

```python

259

import pandas as pd

260

import ibis

261

262

df = pd.DataFrame({'x': [1, 2, 3], 'y': ['a', 'b', 'c']})

263

schema = ibis.infer_schema(df)

264

print(schema) # Schema({'x': int64, 'y': string})

265

```