0
# Built-in Functions Library
1

2
The Flink Scala Table API provides an extensive collection of built-in functions for date/time operations, mathematical calculations, string manipulation, JSON processing, and utility operations.
3

4
## Date and Time Functions
5

6
### Current Date and Time
7

8
```scala { .api }
9
def currentDate(): Expression                    // Current date
10
def currentTime(): Expression                    // Current time  
11
def currentTimestamp(): Expression               // Current timestamp
12
def localTime(): Expression                      // Local time
13
def localTimestamp(): Expression                 // Local timestamp
14
```
15

16
Usage examples:
17
```scala
18
import org.apache.flink.table.api._
19

20
// Current date/time functions
21
val orders = table.select(
22
  $"orderId",
23
  $"amount", 
24
  currentDate() as "processedDate",
25
  currentTimestamp() as "processedTime"
26
)
27

28
// Time-based filtering
29
val recentOrders = table.filter($"orderDate" >= currentDate() - interval(lit(7), DAY))
30
```
31

32
### Date Arithmetic
33

34
```scala { .api }
35
def interval(count: Expression, unit: TimePointUnit): Expression
36
```
37

38
Usage examples:
39
```scala
40
// Date intervals
41
val futureDate = $"startDate" + interval(lit(30), DAY)
42
val pastMonth = currentDate() - interval(lit(1), MONTH)
43
val nextYear = $"birthDate" + interval(lit(1), YEAR)
44

45
// Time intervals  
46
val laterTime = currentTime() + interval(lit(2), HOUR)
47
val earlierTime = $"meetingTime" - interval(lit(15), MINUTE)
48
```
49

50
## Mathematical Functions
51

52
### Constants
53

54
```scala { .api }
55
def pi(): Expression                             // π constant
56
def e(): Expression                              // Euler's number constant
57
```
58

59
### Random Number Generation
60

61
```scala { .api }
62
def rand(): Expression                           // Random double [0.0, 1.0)
63
def randInteger(bound: Expression): Expression   // Random integer [0, bound)
64
```
65

66
### Trigonometric Functions
67

68
```scala { .api }
69
def atan2(y: Expression, x: Expression): Expression  // Arctangent of y/x
70
```
71

72
### Logarithmic and Exponential
73

74
```scala { .api }
75
def log(base: Expression, antilogarithm: Expression): Expression  // Logarithm
76
def exp(base: Expression): Expression                            // Exponential
77
def power(base: Expression, exponent: Expression): Expression    // Power
78
```
79

80
### Arithmetic Functions
81

82
```scala { .api }
83
def mod(numeric1: Expression, numeric2: Expression): Expression  // Modulo
84
```
85

86
Usage examples:
87
```scala
88
// Mathematical calculations
89
val circleArea = pi() * power($"radius", lit(2))
90
val randomScore = rand() * lit(100)  
91
val randomId = randInteger(lit(1000000))
92

93
// Logarithmic operations
94
val logBase10 = log(lit(10), $"value")
95
val naturalLog = log(e(), $"value")
96
val exponential = exp($"growthRate")
97

98
// Trigonometry
99
val angle = atan2($"y", $"x")
100

101
// Modulo operations
102
val evenOdd = mod($"number", lit(2)) === lit(0)
103
```
104

105
## String Functions
106

107
### Basic String Operations
108

109
```scala { .api }
110
def concat(string: Expression*): Expression                           // Concatenate strings
111
def concatWs(separator: Expression, string: Expression*): Expression  // Concatenate with separator
112
def upper(string: Expression): Expression                            // Uppercase
113
def lower(string: Expression): Expression                            // Lowercase
114
def length(string: Expression): Expression                           // String length
115
def position(string: Expression, substring: Expression): Expression  // Substring position
116
```
117

118
### String Generation
119

120
```scala { .api }
121
def uuid(): Expression                           // Generate UUID string
122
```
123

124
Usage examples:
125
```scala
126
// String concatenation
127
val fullName = concat($"firstName", lit(" "), $"lastName")
128
val address = concatWs(lit(", "), $"street", $"city", $"state", $"zip")
129

130
// Case conversion
131
val upperName = upper($"name")
132
val lowerEmail = lower($"email")
133

134
// String analysis
135
val nameLength = length($"productName")
136
val emailAtPosition = position($"email", lit("@"))
137

138
// Unique identifiers
139
val trackingId = uuid()
140

141
// Complex string operations
142
val emailDomain = substring(
143
  $"email", 
144
  position($"email", lit("@")) + lit(1), 
145
  length($"email")
146
)
147
```
148

149
## JSON Functions
150

151
### JSON Construction
152

153
```scala { .api }
154
def jsonString(string: Expression): Expression                     // Create JSON string
155
def jsonObject(keyValue: Expression*): Expression                  // Create JSON object
156
def jsonArray(element: Expression*): Expression                    // Create JSON array
157
```
158

159
### JSON Querying
160

161
```scala { .api }
162
def jsonValue(jsonString: Expression, path: Expression): Expression   // Extract JSON value
163
def jsonQuery(jsonString: Expression, path: Expression): Expression   // Extract JSON fragment
164
```
165

166
Usage examples:
167
```scala
168
// JSON construction
169
val userJson = jsonObject(
170
  lit("id"), $"userId",
171
  lit("name"), $"userName", 
172
  lit("email"), $"userEmail"
173
)
174

175
val tagsArray = jsonArray($"tag1", $"tag2", $"tag3")
176

177
// JSON querying
178
val jsonData = lit("""{"user": {"name": "John", "age": 30}, "active": true}""")
179
val userName = jsonValue(jsonData, lit("$.user.name"))
180
val userInfo = jsonQuery(jsonData, lit("$.user"))
181

182
// Working with JSON columns
183
val extractedAge = jsonValue($"userProfile", lit("$.age"))
184
val preferences = jsonQuery($"settings", lit("$.preferences"))
185
```
186

187
## Utility Functions
188

189
### Null Handling
190

191
```scala { .api }
192
def nullOf(dataType: DataType): Expression                         // Create typed null
193
def coalesce(expr: Expression*): Expression                        // First non-null value
194
def isnull(expr: Expression): Expression                          // Check if null
195
def isNotNull(expr: Expression): Expression                       // Check if not null
196
```
197

198
### Conditional Logic
199

200
```scala { .api }
201
def ifThenElse(
202
  condition: Expression, 
203
  ifTrue: Expression, 
204
  ifFalse: Expression
205
): Expression                                                      // Conditional expression
206
```
207

208
Usage examples:
209
```scala
210
// Null handling
211
val defaultName = coalesce($"nickname", $"firstName", lit("Anonymous"))
212
val hasEmail = isNotNull($"email")
213

214
// Conditional logic
215
val status = ifThenElse(
216
  $"age" >= lit(18),
217
  lit("Adult"),
218
  lit("Minor")
219
)
220

221
val discountedPrice = ifThenElse(
222
  $"memberType" === lit("PREMIUM"),
223
  $"price" * lit(0.9),
224
  $"price"
225
)
226

227
// Typed nulls
228
val nullString = nullOf(DataTypes.STRING())
229
val nullInt = nullOf(DataTypes.INT())
230
```
231

232
## Column Operations
233

234
### Column Selection and Manipulation
235

236
```scala { .api }
237
def withColumns(columns: Expression*): Expression      // Add/modify columns
238
def withoutColumns(columns: Expression*): Expression   // Remove columns
239
def range(from: Expression, to: Expression): Expression // Column range
240
```
241

242
Usage examples:
243
```scala
244
// Add computed columns
245
val enrichedData = withColumns(
246
  $"totalPrice" := $"price" * $"quantity",
247
  $"discountedPrice" := $"price" * (lit(1) - $"discountRate"),
248
  $"category" := upper($"productType")
249
)
250

251
// Remove columns
252
val publicData = withoutColumns($"ssn", $"creditCard", $"internalId")
253

254
// Column ranges (for wide tables)
255
val subset = range($"col1", $"col10")  // Select col1 through col10
256
```
257

258
## Logical Operations
259

260
### Boolean Functions
261

262
```scala { .api }
263
def and(left: Expression, right: Expression): Expression    // Logical AND
264
def or(left: Expression, right: Expression): Expression     // Logical OR
265
def not(expression: Expression): Expression                 // Logical NOT
266
```
267

268
Usage examples:
269
```scala
270
// Explicit logical operations (alternative to operators)
271
val condition1 = and($"age" >= lit(18), $"hasLicense" === lit(true))
272
val condition2 = or($"priority" === lit("HIGH"), $"urgent" === lit(true))
273
val condition3 = not($"deleted" === lit(true))
274

275
// Complex boolean logic
276
val eligibleForDiscount = and(
277
  or($"memberType" === lit("GOLD"), $"memberType" === lit("PLATINUM")),
278
  and($"totalPurchases" > lit(1000), not($"suspended" === lit(true)))
279
)
280
```
281

282
## Advanced Function Usage
283

284
### Function Composition
285

286
```scala
287
// Combining multiple functions
288
val processedEmail = lower(trim($"email"))
289
val formattedName = concat(
290
  upper(substring($"firstName", lit(1), lit(1))),
291
  lower(substring($"firstName", lit(2), length($"firstName")))
292
)
293

294
// Complex date calculations  
295
val businessDaysUntilDeadline = ifThenElse(
296
  dayOfWeek(currentDate()) <= lit(5),  // Monday-Friday
297
  ($"deadline" - currentDate()) * lit(5) / lit(7),
298
  ($"deadline" - currentDate()) * lit(5) / lit(7)
299
)
300
```
301

302
### Type-Safe Function Calls
303

304
```scala
305
// Explicit type casting when needed
306
val stringToInt = cast($"numberString", DataTypes.INT())
307
val doubleToDecimal = cast($"floatValue", DataTypes.DECIMAL(10, 2))
308

309
// Type-aware operations
310
val numericComparison = cast($"stringNumber", DataTypes.DOUBLE()) > lit(100.0)
311
```
312

313
### Custom Function Integration
314

315
```scala
316
// Calling custom scalar functions
317
class CalculateTax extends ScalarFunction {
318
  def eval(amount: Double, rate: Double): Double = amount * rate
319
}
320

321
val taxFunction = new CalculateTax()
322
val totalWithTax = call(taxFunction, $"price", $"taxRate")
323

324
// Calling catalog functions
325
val customResult = call("MY_CUSTOM_FUNCTION", $"input1", $"input2")
326
```
327

328
## Performance Considerations
329

330
### Function Optimization
331

332
- Built-in functions are optimized at query planning time
333
- Constant expressions are evaluated once during planning
334
- Common subexpressions are automatically eliminated
335
- Function calls are vectorized when possible
336

337
### Memory Efficiency
338

339
- String functions use efficient buffer management
340
- JSON functions utilize streaming parsers for large documents
341
- Mathematical functions leverage CPU optimizations
342
- Date/time functions use optimized calendar implementations
343

344
## Best Practices
345

346
### Null Safety
347

348
```scala
349
// Always consider null handling
350
val safeLength = ifThenElse(
351
  isNotNull($"text"),
352
  length($"text"),
353
  lit(0)
354
)
355

356
// Use coalesce for default values
357
val displayName = coalesce($"preferredName", $"firstName", lit("Unknown"))
358
```
359

360
### Type Consistency
361

362
```scala
363
// Ensure compatible types in operations
364
val numericResult = cast($"stringValue", DataTypes.DOUBLE()) + lit(10.0)
365

366
// Consistent null handling across types
367
val nullSafeResult = coalesce($"optionalValue", nullOf(DataTypes.STRING()))
368
```
369

370
### Performance Optimization
371

372
```scala
373
// Prefer built-in functions over complex expressions
374
val optimizedCase = ifThenElse($"status" === lit("ACTIVE"), lit(1), lit(0))
375

376
// Rather than complex string manipulation
377
val efficientExtraction = jsonValue($"data", lit("$.user.id"))
378
// Instead of multiple string operations
379
```
380

381
### Readable Function Composition
382

383
```scala
384
// Break complex function chains into readable parts
385
val cleanEmail = lower(trim($"rawEmail"))
386
val isValidEmail = position(cleanEmail, lit("@")) > lit(0)
387
val processedEmail = ifThenElse(isValidEmail, cleanEmail, lit("invalid"))
388
```