or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

async.mdexternal-systems.mdindex.mditeration.mdjoins.mdmachine-learning.mdside-output.mdsocket.mdutilities.mdwindowing.mdwordcount.md
tile.json

tessl/maven-org-apache-flink--flink-examples-streaming-2-10

Apache Flink streaming examples demonstrating various stream processing patterns and use cases

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
mavenpkg:maven/org.apache.flink/flink-examples-streaming_2.10@1.3.x

To install, run

npx @tessl/cli install tessl/maven-org-apache-flink--flink-examples-streaming-2-10@1.3.0

index.mddocs/

Apache Flink Streaming Examples

Apache Flink streaming examples demonstrating various stream processing patterns and use cases. This collection provides comprehensive reference implementations showcasing Flink's DataStream API capabilities, real-time stream processing patterns, and integration with external systems.

Package Information

  • Package Name: flink-examples-streaming_2.10
  • Package Type: Maven
  • Language: Java/Scala
  • Maven Coordinates: org.apache.flink:flink-examples-streaming_2.10:1.3.3
  • Installation: Include as dependency in Maven projects or use standalone JARs

Core Dependencies

<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-examples-streaming_2.10</artifactId>
    <version>1.3.3</version>
</dependency>

Basic Usage

These examples are standalone executable programs that demonstrate various streaming patterns:

# Run examples using Flink CLI
$FLINK_HOME/bin/flink run WordCount.jar --input input.txt --output output.txt

# Or execute directly with Java
java -cp flink-examples-streaming_2.10-1.3.3.jar \
  org.apache.flink.streaming.examples.wordcount.WordCount \
  --input input.txt --output output.txt

Architecture

The examples are organized into functional categories, each demonstrating specific aspects of Flink streaming:

  • Core Patterns: Basic streaming operations, transformations, and aggregations
  • Windowing: Time-based and count-based window operations with custom triggers
  • State Management: Stateful operations, checkpointing, and exactly-once processing
  • External Integration: Connectors for Kafka, Twitter, and socket-based data sources
  • Advanced Patterns: Iterations, joins, async I/O, and machine learning workflows

Each example is self-contained with configurable parameters and includes both Java and Scala implementations where applicable.

Capabilities

Word Count Examples

Basic streaming word count implementations demonstrating fundamental DataStream operations and tuple-based processing.

// Main executable classes
public class WordCount {
    public static void main(String[] args) throws Exception;
    
    public static final class Tokenizer 
        implements FlatMapFunction<String, Tuple2<String, Integer>> {
        public void flatMap(String value, Collector<Tuple2<String, Integer>> out) 
            throws Exception;
    }
}

public class PojoExample {
    public static void main(String[] args) throws Exception;
}

Word Count Examples

Socket Streaming Examples

Real-time data processing from socket connections with windowing and aggregation operations.

public class SocketWindowWordCount {
    public static void main(String[] args) throws Exception;
    
    public static class WordWithCount {
        public String word;
        public long count;
        public WordWithCount();
        public WordWithCount(String word, long count);
        public String toString();
    }
}

Socket Streaming

Windowing Examples

Advanced windowing patterns including time windows, session windows, and custom triggers with event-time processing.

public class TopSpeedWindowing {
    public static void main(String[] args) throws Exception;
}

public class SessionWindowing {
    public static void main(String[] args) throws Exception;
}

public class WindowWordCount {
    public static void main(String[] args) throws Exception;
}

Windowing Patterns

Side Output Examples

Advanced stream processing with conditional routing using ProcessFunction and OutputTag for stream splitting.

public class SideOutputExample {
    static final OutputTag<String> rejectedWordsTag = new OutputTag<String>("rejected") {};
    public static void main(String[] args) throws Exception;
    
    public static final class Tokenizer extends ProcessFunction<String, Tuple2<String, Integer>> {
        public void processElement(String value, Context ctx, Collector<Tuple2<String, Integer>> out) 
            throws Exception;
    }
}

Side Output Processing

Asynchronous I/O Examples

Non-blocking external system integration with configurable parallelism and error handling.

public class AsyncIOExample {
    public static void main(String[] args) throws Exception;
}

Asynchronous I/O

Iteration Examples

Streaming iterations with feedback loops and convergence criteria for iterative algorithms.

public class IterateExample {
    public static void main(String[] args) throws Exception;
    
    public static class InputMap 
        implements MapFunction<Tuple2<Integer, Integer>, 
                              Tuple5<Integer, Integer, Integer, Integer, Integer>>;
    
    public static class Step 
        implements MapFunction<Tuple5<Integer, Integer, Integer, Integer, Integer>, 
                              Tuple5<Integer, Integer, Integer, Integer, Integer>>;
    
    public static class MySelector 
        implements OutputSelector<Tuple5<Integer, Integer, Integer, Integer, Integer>>;
}

Streaming Iterations

Stream Join Examples

Time-based stream joins with coordinated watermarks and window-based join operations.

public class WindowJoin {
    public static void main(String[] args) throws Exception;
}

Stream Joins

External System Integration

Integration examples for Kafka, Twitter, and other external data sources with fault-tolerant connectors.

public class ReadFromKafka {
    public static void main(String[] args) throws Exception;
}

public class WriteIntoKafka {
    public static void main(String[] args) throws Exception;
}

public class TwitterExample {
    public static void main(String[] args) throws Exception;
}

External Systems

Machine Learning Examples

Incremental learning patterns and online algorithm implementations for streaming ML workflows.

public class IncrementalLearningSkeleton {
    public static void main(String[] args) throws Exception;
}

Machine Learning

Utility Classes

Shared utility classes and data generators used across multiple examples.

public class ThrottledIterator<T> implements Iterator<T>, Serializable {
    public ThrottledIterator(Iterator<T> source, long elementsPerSecond);
    public boolean hasNext();
    public T next();
    public void remove(); // throws UnsupportedOperationException
}

Utilities

Common Types

// Flink core types used throughout examples
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.api.java.utils.ParameterTool;
import org.apache.flink.util.Collector;
import org.apache.flink.streaming.api.functions.ProcessFunction;
import org.apache.flink.util.OutputTag;

// Common data structures
class Tuple2<T0, T1> {
    public T0 f0;
    public T1 f1;
    public Tuple2();
    public Tuple2(T0 f0, T1 f1);
}

class Tuple4<T0, T1, T2, T3> {
    public T0 f0;
    public T1 f1; 
    public T2 f2;
    public T3 f3;
    public Tuple4();
    public Tuple4(T0 f0, T1 f1, T2 f2, T3 f3);
}

class Tuple5<T0, T1, T2, T3, T4> {
    public T0 f0;
    public T1 f1;
    public T2 f2;
    public T3 f3;
    public T4 f4;
    public Tuple5();
    public Tuple5(T0 f0, T1 f1, T2 f2, T3 f3, T4 f4);
}

// Advanced stream processing types
abstract class ProcessFunction<I, O> extends AbstractRichFunction {
    public abstract void processElement(I value, Context ctx, Collector<O> out) throws Exception;
    public void onTimer(long timestamp, OnTimerContext ctx, Collector<O> out) throws Exception;
    
    public abstract class Context {
        public abstract <X> void output(OutputTag<X> outputTag, X value);
        public abstract long timestamp();
        public abstract long currentWatermark();
    }
}

class OutputTag<T> {
    public OutputTag(String id);
}