or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

aggregation-grouping.mdcatalog-management.mdexpressions.mdindex.mdsql-integration.mdtable-environment.mdtable-operations.mduser-defined-functions.mdwindow-operations.md
tile.json

tessl/maven-org-apache-flink--flink-table-api-java

Java API for Apache Flink's Table ecosystem, enabling type-safe table operations and SQL query execution.

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
mavenpkg:maven/org.apache.flink/flink-table-api-java@2.1.x

To install, run

npx @tessl/cli install tessl/maven-org-apache-flink--flink-table-api-java@2.1.0

index.mddocs/

Apache Flink Table API Java

Apache Flink Table API Java provides a unified table-centric programming interface for both batch and streaming data processing. It offers a comprehensive ecosystem for creating, manipulating, and querying tables with type-safe Java APIs, SQL support, and seamless integration with Flink's distributed processing engine.

Package Information

  • Package Name: flink-table-api-java
  • Package Type: maven
  • Language: Java
  • Installation: 
    <dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-table-api-java</artifactId>
  <version>2.1.0</version>
</dependency>

Core Imports

import org.apache.flink.table.api.*;
import org.apache.flink.table.expressions.Expression;
import static org.apache.flink.table.api.Expressions.*;

For catalog operations:

import org.apache.flink.table.catalog.*;

For user-defined functions:

import org.apache.flink.table.functions.*;

Basic Usage

import org.apache.flink.table.api.*;
import static org.apache.flink.table.api.Expressions.*;

// Create table environment
EnvironmentSettings settings = EnvironmentSettings
    .newInstance()
    .inStreamingMode()
    .build();
TableEnvironment tableEnv = TableEnvironment.create(settings);

// Create table from source
Table sourceTable = tableEnv.from("input_table");

// Perform transformations
Table result = sourceTable
    .select($("name"), $("age"), $("salary"))
    .filter($("age").isGreater(25))
    .groupBy($("name"))
    .select($("name"), $("age").max(), $("salary").avg());

// Execute and collect results
TableResult tableResult = result.execute();
tableResult.print();

// SQL alternative
Table sqlResult = tableEnv.sqlQuery(
    "SELECT name, MAX(age), AVG(salary) " +
    "FROM input_table " +
    "WHERE age > 25 " +
    "GROUP BY name"
);

Architecture

The Flink Table API is built around several key components:

  • TableEnvironment: Central entry point for all table operations, catalog management, and SQL execution
  • Table: Core abstraction representing data transformation pipelines with lazy evaluation
  • Expression System: Type-safe column references and function calls for table operations
  • Catalog System: Metadata management for tables, functions, and data sources
  • SQL Integration: Full SQL support with query parsing and execution
  • Window Operations: Time-based and count-based windowing for streaming aggregations
  • User-Defined Functions: Extensible function system for custom logic

Capabilities

Table Environment and Setup

Core functionality for creating and configuring table environments, managing catalogs, and establishing execution contexts.

public interface TableEnvironment {
    static TableEnvironment create(EnvironmentSettings settings);
    
    Table from(String path);
    TableResult executeSql(String statement);
    Table sqlQuery(String query);
    
    void createTable(String path, TableDescriptor descriptor);
    void createTemporaryView(String path, Table view);
}

public class EnvironmentSettings {
    public static Builder newInstance();
    
    public static class Builder {
        public Builder inStreamingMode();
        public Builder inBatchMode();
        public EnvironmentSettings build();
    }
}

Table Environment and Setup

Table Operations and Transformations

Core table transformation operations including selection, filtering, joining, and data manipulation with type-safe expression handling.

public interface Table extends Explainable<Table>, Executable {
    Table select(Expression... fields);
    Table filter(Expression predicate);
    Table join(Table right, Expression joinPredicate);
    Table leftOuterJoin(Table right, Expression joinPredicate);
    
    GroupedTable groupBy(Expression... fields);
    Table orderBy(Expression... fields);
    Table limit(int fetch);
    
    TableResult execute();
    TableResult executeInsert(String tablePath);
}

Table Operations and Transformations

Expressions and Column References

Type-safe expression system for building column references, function calls, and complex predicates in table operations.

public final class Expressions {
    public static Expression $(String name);
    public static Expression lit(Object value);
    public static Expression call(String name, Object... args);
    
    // Comparison operators
    public static Expression isEqual(Expression left, Expression right);
    public static Expression isGreater(Expression left, Expression right);
    public static Expression isLess(Expression left, Expression right);
    
    // Logical operators  
    public static Expression and(Expression left, Expression right);
    public static Expression or(Expression left, Expression right);
    public static Expression not(Expression expression);
}

public interface Expression {
    Expression as(String alias);
    Expression isEqual(Object other);
    Expression isGreater(Object other);
    Expression plus(Object other);
    Expression minus(Object other);
}

Expressions and Column References

Aggregation and Grouping

Aggregation operations with support for grouping, window functions, and advanced aggregation patterns for both batch and streaming scenarios.

public interface GroupedTable {
    Table select(Expression... fields);
    AggregatedTable aggregate(Expression aggregateFunction);
    FlatAggregateTable flatAggregate(Expression tableAggregateFunction);
}

public interface AggregatedTable {
    Table select(Expression... fields);
}

public interface FlatAggregateTable {
    Table select(Expression... fields);
}

Aggregation and Grouping

Window Operations

Time-based and count-based windowing operations for streaming data processing, including tumbling, sliding, and session windows.

public abstract class GroupWindow {
    public abstract GroupWindow as(String alias);
}

public final class Tumble {
    public static TumbleWithSize over(Expression size);
}

public final class Slide {
    public static SlideWithSize over(Expression size);
}

public final class Session {
    public static SessionWithGap withGap(Expression gap);
}

public interface WindowGroupedTable {
    Table select(Expression... fields);
}

Window Operations

Catalog and Metadata Management

Catalog system for managing table metadata, data sources, functions, and multi-catalog environments with persistent storage.

public class CatalogManager {
    public void registerCatalog(String catalogName, Catalog catalog);
    public Optional<Catalog> getCatalog(String catalogName);
    public void setCurrentCatalog(String catalogName);
    public void setCurrentDatabase(String databaseName);
}

public interface Catalog {
    void createDatabase(String name, CatalogDatabase database, boolean ignoreIfExists);
    void createTable(ObjectPath tablePath, CatalogTable table, boolean ignoreIfExists);
    CatalogTable getTable(ObjectPath tablePath);
}

Catalog and Metadata Management

User-Defined Functions

Framework for creating and registering custom scalar, table, and aggregate functions to extend the built-in function library.

public abstract class ScalarFunction extends UserDefinedFunction {
    // User implements eval() methods with various signatures
}

public abstract class TableFunction<T> extends UserDefinedFunction {
    public void eval(Object... args);
    protected void collect(T result);
}

public abstract class AggregateFunction<T, ACC> extends UserDefinedFunction {
    public ACC createAccumulator();
    public void accumulate(ACC accumulator, Object... args);
    public T getValue(ACC accumulator);
}

User-Defined Functions

SQL Integration

Native SQL support with query parsing, execution planning, and seamless integration between Table API and SQL operations.

public interface TableEnvironment {
    Table sqlQuery(String query);
    TableResult executeSql(String statement);
    void executeSql(String statement);
    
    StatementSet createStatementSet();
}

public interface StatementSet {
    StatementSet addInsertSql(String statement);
    TableResult execute();
}

SQL Integration

Types

Core Types

public interface TableResult {
    void print();
    CloseableIterator<Row> collect();
    ResultKind getResultKind();
    TableSchema getTableSchema();
}

public enum ResultKind {
    SUCCESS,
    SUCCESS_WITH_INFO
}

public final class TableConfig implements WritableConfig, ReadableConfig {
    public Configuration getConfiguration();
    public void setSqlDialect(SqlDialect dialect);
}

public enum SqlDialect {
    DEFAULT,
    HIVE
}

public class TableDescriptor {
    public static Builder forConnector(String connector);
    
    public static class Builder {
        public Builder schema(Schema schema);
        public Builder option(String key, String value);
        public TableDescriptor build();
    }
}

Exception Types

public class SqlParserException extends RuntimeException {
    public SqlParserException(String message);
    public SqlParserException(String message, Throwable cause);
}

public class SqlParserEOFException extends SqlParserException {
    public SqlParserEOFException(String message);
}