tessl/maven-org-apache-flink--flink-examples-streaming-2-10

Apache Flink streaming examples demonstrating various stream processing patterns and use cases

—

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Securityby

Pending

The risk profile of this skill

Overview

Eval results

Files

Apache Flink Streaming Examples

Name: tessl/maven-org-apache-flink--flink-examples-streaming-2-10
Author: tessl

Apache Flink streaming examples demonstrating various stream processing patterns and use cases. This collection provides comprehensive reference implementations showcasing Flink's DataStream API capabilities, real-time stream processing patterns, and integration with external systems.

Package Information

Package Name: flink-examples-streaming_2.10
Package Type: Maven
Language: Java/Scala
Maven Coordinates: org.apache.flink:flink-examples-streaming_2.10:1.3.3
Installation: Include as dependency in Maven projects or use standalone JARs

Core Dependencies

<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-examples-streaming_2.10</artifactId>
    <version>1.3.3</version>
</dependency>

Basic Usage

These examples are standalone executable programs that demonstrate various streaming patterns:

# Run examples using Flink CLI
$FLINK_HOME/bin/flink run WordCount.jar --input input.txt --output output.txt

# Or execute directly with Java
java -cp flink-examples-streaming_2.10-1.3.3.jar \
  org.apache.flink.streaming.examples.wordcount.WordCount \
  --input input.txt --output output.txt

Architecture

The examples are organized into functional categories, each demonstrating specific aspects of Flink streaming:

Core Patterns: Basic streaming operations, transformations, and aggregations
Windowing: Time-based and count-based window operations with custom triggers
State Management: Stateful operations, checkpointing, and exactly-once processing
External Integration: Connectors for Kafka, Twitter, and socket-based data sources
Advanced Patterns: Iterations, joins, async I/O, and machine learning workflows

Each example is self-contained with configurable parameters and includes both Java and Scala implementations where applicable.

Capabilities

Word Count Examples

Basic streaming word count implementations demonstrating fundamental DataStream operations and tuple-based processing.

// Main executable classes
public class WordCount {
    public static void main(String[] args) throws Exception;
    
    public static final class Tokenizer 
        implements FlatMapFunction<String, Tuple2<String, Integer>> {
        public void flatMap(String value, Collector<Tuple2<String, Integer>> out) 
            throws Exception;
    }
}

public class PojoExample {
    public static void main(String[] args) throws Exception;
}

Word Count Examples

Socket Streaming Examples

Real-time data processing from socket connections with windowing and aggregation operations.

public class SocketWindowWordCount {
    public static void main(String[] args) throws Exception;
    
    public static class WordWithCount {
        public String word;
        public long count;
        public WordWithCount();
        public WordWithCount(String word, long count);
        public String toString();
    }
}

Socket Streaming

Windowing Examples

Advanced windowing patterns including time windows, session windows, and custom triggers with event-time processing.

public class TopSpeedWindowing {
    public static void main(String[] args) throws Exception;
}

public class SessionWindowing {
    public static void main(String[] args) throws Exception;
}

public class WindowWordCount {
    public static void main(String[] args) throws Exception;
}

Windowing Patterns

Side Output Examples

Advanced stream processing with conditional routing using ProcessFunction and OutputTag for stream splitting.

public class SideOutputExample {
    static final OutputTag<String> rejectedWordsTag = new OutputTag<String>("rejected") {};
    public static void main(String[] args) throws Exception;
    
    public static final class Tokenizer extends ProcessFunction<String, Tuple2<String, Integer>> {
        public void processElement(String value, Context ctx, Collector<Tuple2<String, Integer>> out) 
            throws Exception;
    }
}

Side Output Processing

Asynchronous I/O Examples

Non-blocking external system integration with configurable parallelism and error handling.

public class AsyncIOExample {
    public static void main(String[] args) throws Exception;
}

Asynchronous I/O

Iteration Examples

Streaming iterations with feedback loops and convergence criteria for iterative algorithms.

public class IterateExample {
    public static void main(String[] args) throws Exception;
    
    public static class InputMap 
        implements MapFunction<Tuple2<Integer, Integer>, 
                              Tuple5<Integer, Integer, Integer, Integer, Integer>>;
    
    public static class Step 
        implements MapFunction<Tuple5<Integer, Integer, Integer, Integer, Integer>, 
                              Tuple5<Integer, Integer, Integer, Integer, Integer>>;
    
    public static class MySelector 
        implements OutputSelector<Tuple5<Integer, Integer, Integer, Integer, Integer>>;
}

Streaming Iterations

Stream Join Examples

Time-based stream joins with coordinated watermarks and window-based join operations.

public class WindowJoin {
    public static void main(String[] args) throws Exception;
}

Stream Joins

External System Integration

Integration examples for Kafka, Twitter, and other external data sources with fault-tolerant connectors.

public class ReadFromKafka {
    public static void main(String[] args) throws Exception;
}

public class WriteIntoKafka {
    public static void main(String[] args) throws Exception;
}

public class TwitterExample {
    public static void main(String[] args) throws Exception;
}

External Systems

Machine Learning Examples

Incremental learning patterns and online algorithm implementations for streaming ML workflows.

public class IncrementalLearningSkeleton {
    public static void main(String[] args) throws Exception;
}

Machine Learning

Utility Classes

Shared utility classes and data generators used across multiple examples.

public class ThrottledIterator<T> implements Iterator<T>, Serializable {
    public ThrottledIterator(Iterator<T> source, long elementsPerSecond);
    public boolean hasNext();
    public T next();
    public void remove(); // throws UnsupportedOperationException
}

Utilities

Common Types

// Flink core types used throughout examples
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.api.java.utils.ParameterTool;
import org.apache.flink.util.Collector;
import org.apache.flink.streaming.api.functions.ProcessFunction;
import org.apache.flink.util.OutputTag;

// Common data structures
class Tuple2<T0, T1> {
    public T0 f0;
    public T1 f1;
    public Tuple2();
    public Tuple2(T0 f0, T1 f1);
}

class Tuple4<T0, T1, T2, T3> {
    public T0 f0;
    public T1 f1; 
    public T2 f2;
    public T3 f3;
    public Tuple4();
    public Tuple4(T0 f0, T1 f1, T2 f2, T3 f3);
}

class Tuple5<T0, T1, T2, T3, T4> {
    public T0 f0;
    public T1 f1;
    public T2 f2;
    public T3 f3;
    public T4 f4;
    public Tuple5();
    public Tuple5(T0 f0, T1 f1, T2 f2, T3 f3, T4 f4);
}

// Advanced stream processing types
abstract class ProcessFunction<I, O> extends AbstractRichFunction {
    public abstract void processElement(I value, Context ctx, Collector<O> out) throws Exception;
    public void onTimer(long timestamp, OnTimerContext ctx, Collector<O> out) throws Exception;
    
    public abstract class Context {
        public abstract <X> void output(OutputTag<X> outputTag, X value);
        public abstract long timestamp();
        public abstract long currentWatermark();
    }
}

class OutputTag<T> {
    public OutputTag(String id);
}

Workspace: tessl
Visibility: Public
Created: 6 months ago
Last updated: 2 months ago
Describes: pkg:maven/org.apache.flink/flink-examples-streaming_2.10@1.3.x
Publish Source: CLI
Badge