Apache Flink Table Runtime

Apache Flink Table Runtime provides the essential execution infrastructure for Flink's Table API and SQL operations. This library contains classes required by task managers to execute table programs, serving as the foundational runtime layer that bridges high-level SQL and Table API constructs with Flink's distributed streaming execution engine.

Package Information

• Package Name: flink-table-runtime
• Package Type: maven
• Group ID: org.apache.flink
• Language: Java
• Installation: Include as dependency in Maven/Gradle project

<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-table-runtime</artifactId>
    <version>2.1.0</version>
</dependency>

Core Imports

Most functionality is internal to Flink's runtime, but the public APIs are:

import org.apache.flink.formats.raw.RawFormatOptions;
import org.apache.flink.table.runtime.typeutils.SortedMapTypeInfo;

Basic Usage

This library is primarily intended for internal use by Flink's table runtime system. External usage is limited but includes:

import org.apache.flink.formats.raw.RawFormatOptions;
import org.apache.flink.table.runtime.typeutils.SortedMapTypeInfo;
import org.apache.flink.api.common.typeinfo.TypeInformation;
import java.util.Comparator;
import java.util.SortedMap;

// Using raw format options for configuration
String endianness = RawFormatOptions.BIG_ENDIAN;
ConfigOption<String> charsetOption = RawFormatOptions.CHARSET;

// Creating type information for sorted maps
SortedMapTypeInfo<String, Integer> mapTypeInfo = new SortedMapTypeInfo<>(
    TypeInformation.of(String.class),
    TypeInformation.of(Integer.class),
    String.CASE_INSENSITIVE_ORDER
);

Architecture

The module is organized into these main functional areas:

• Data Layer - Internal data structures, binary formats, and converters for efficient data processing
• Functions Layer - Runtime function implementations for SQL operations and expressions
• Operators Layer - Stream processing operators for table operations and transformations
• Formats Layer - Data format support, currently including raw byte format processing
• Utilities Layer - Runtime utilities, type system components, and serialization support

This design enables efficient processing of both bounded and unbounded tabular datasets in distributed environments, providing the runtime foundation for Flink's unified batch and stream processing capabilities.

Capabilities

Raw Format Configuration

Configuration options for raw byte-based data format processing with configurable endianness and character encoding.

public class RawFormatOptions {
    public static final String BIG_ENDIAN = "big-endian";
    public static final String LITTLE_ENDIAN = "little-endian";
    public static final ConfigOption<String> ENDIANNESS;
    public static final ConfigOption<String> CHARSET;
}

Type System Support

Type information and utilities for sorted maps and custom type serialization.

public class SortedMapTypeInfo<K, V> extends AbstractMapTypeInfo<K, V, SortedMap<K, V>> {
    public SortedMapTypeInfo(
        TypeInformation<K> keyTypeInfo,
        TypeInformation<V> valueTypeInfo,
        Comparator<K> keyComparator
    );
    
    public SortedMapTypeInfo(
        Class<K> keyClass,
        Class<V> valueClass,
        Comparator<K> keyComparator
    );
    
    public SortedMapTypeInfo(Class<K> keyClass, Class<V> valueClass);
    
    public Class<SortedMap<K, V>> getTypeClass();
    public TypeSerializer<SortedMap<K, V>> createSerializer(SerializerConfig config);
    public boolean canEqual(Object obj);
}

Types

// Core configuration types
public class ConfigOption<T> {
    // Configuration option for various format settings
}

// Flink type system
public interface TypeInformation<T> {
    // Type information for Flink's serialization system
}

// Comparator interface
public interface Comparator<T> {
    // Standard Java comparator interface
}

// Map interface  
public interface SortedMap<K, V> extends Map<K, V> {
    // Standard Java sorted map interface
}

Important Notes

Limited Public API: This library has minimal public API surface with only 2 classes marked @PublicEvolving (RawFormatOptions and SortedMapTypeInfo). Most functionality is marked @Internal and intended for use within Flink's runtime system.

Runtime Focus: The primary purpose is to provide execution infrastructure for Flink's Table API and SQL operations. Direct external usage is limited and primarily involves configuration and type system integration.

Internal Implementation: Most classes in this module serve as internal runtime infrastructure. The public APIs documented here represent the only external interfaces intended for use outside of Flink's core runtime system.

Dependencies: Requires core Flink table modules for proper integration with Flink's table processing ecosystem.