CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/maven-org-apache-flink--flink-walkthrough-common-2-12

Apache Flink walkthrough common library containing shared entities and utilities for Flink streaming examples

—

Pending
Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Pending

The risk profile of this skill

Overview
Eval results
Files

index.mddocs/

Flink Walkthrough Common

Apache Flink walkthrough common library containing shared entities and utilities for Flink streaming examples. This package provides Transaction and Alert data classes, input/output formatters, and source/sink implementations designed to support educational walkthroughs and example applications demonstrating Flink's stream processing capabilities.

Package Information

  • Package Name: flink-walkthrough-common_2.12
  • Package Type: maven
  • Language: Java
  • Group ID: org.apache.flink
  • Artifact ID: flink-walkthrough-common_2.12
  • Installation: Add to pom.xml: 
    <dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-walkthrough-common_2.12</artifactId>
  <version>1.14.6</version>
</dependency>

Core Imports

import org.apache.flink.walkthrough.common.entity.Transaction;
import org.apache.flink.walkthrough.common.entity.Alert;
import org.apache.flink.walkthrough.common.source.TransactionSource;
import org.apache.flink.walkthrough.common.source.TransactionRowInputFormat;
import org.apache.flink.walkthrough.common.sink.AlertSink;
import org.apache.flink.walkthrough.common.sink.LoggerOutputFormat;

Basic Usage

import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.walkthrough.common.entity.Transaction;
import org.apache.flink.walkthrough.common.entity.Alert;
import org.apache.flink.walkthrough.common.source.TransactionSource;
import org.apache.flink.walkthrough.common.sink.AlertSink;

// Create streaming environment
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

// Create transaction stream with built-in sample data
DataStream<Transaction> transactions = env.addSource(new TransactionSource())
    .name("transactions");

// Process transactions to generate alerts
DataStream<Alert> alerts = transactions
    .filter(transaction -> transaction.getAmount() > 1000.0)
    .map(transaction -> {
        Alert alert = new Alert();
        alert.setId(transaction.getAccountId());
        return alert;
    });

// Output alerts using provided sink
alerts.addSink(new AlertSink()).name("alerts");

env.execute("Fraud Detection");

Capabilities

Entity Classes

Core data classes representing transaction and alert events for streaming applications.

Transaction Entity

Represents a financial transaction with account ID, timestamp, and amount.

public final class Transaction {
    public Transaction();
    public Transaction(long accountId, long timestamp, double amount);
    
    public long getAccountId();
    public void setAccountId(long accountId);
    
    public long getTimestamp();
    public void setTimestamp(long timestamp);
    
    public double getAmount();
    public void setAmount(double amount);
    
    @Override
    public boolean equals(Object o);
    
    @Override
    public int hashCode();
    
    @Override
    public String toString();
}

Alert Entity

Represents an alert event with an identifier.

public final class Alert {
    public Alert();
    
    public long getId();
    public void setId(long id);
    
    @Override
    public boolean equals(Object o);
    
    @Override
    public int hashCode();
    
    @Override
    public String toString();
}

Data Sources

Source functions for generating transaction streams in Flink applications.

Transaction Source

Streaming source that generates transactions with rate limiting for realistic simulation.

public class TransactionSource extends FromIteratorFunction<Transaction> {
    public TransactionSource();
}

Usage:

import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.datastream.DataStream;

StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
DataStream<Transaction> transactions = env.addSource(new TransactionSource());

The TransactionSource generates an endless stream of sample transaction data with a built-in rate limit (100ms delay between transactions) and predefined transaction amounts across 5 different account IDs.

Transaction Row Input Format

Batch input format for reading transactions as Flink Row objects.

public class TransactionRowInputFormat extends GenericInputFormat<Row> 
        implements NonParallelInput {
    @Override
    public void open(GenericInputSplit split);
    
    @Override
    public boolean reachedEnd();
    
    @Override
    public Row nextRecord(Row reuse);
}

Usage:

import org.apache.flink.api.java.ExecutionEnvironment;
import org.apache.flink.api.java.DataSet;
import org.apache.flink.types.Row;

ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
DataSet<Row> transactionRows = env.createInput(new TransactionRowInputFormat());

Data Sinks

Sink functions for outputting processed data from Flink streams.

Alert Sink

Streaming sink that logs alert events to standard output.

public class AlertSink implements SinkFunction<Alert> {
    @Override
    public void invoke(Alert value, Context context);
}

Usage:

import org.apache.flink.streaming.api.datastream.DataStream;

// Assuming you have a DataStream<Alert>
DataStream<Alert> alerts = // ... your alert stream
alerts.addSink(new AlertSink());

Logger Output Format

Batch output format that logs string records at INFO level.

public class LoggerOutputFormat implements OutputFormat<String> {
    @Override
    public void configure(Configuration parameters);
    
    @Override
    public void open(int taskNumber, int numTasks);
    
    @Override
    public void writeRecord(String record);
    
    @Override
    public void close();
}

Usage:

import org.apache.flink.api.java.DataSet;

// Assuming you have a DataSet<String> 
DataSet<String> dataSet = // ... your data
dataSet.output(new LoggerOutputFormat());

Architecture

The library follows Flink's standard patterns and integrates with core Flink interfaces:

  • Entity Classes: Serializable POJOs suitable for Flink's type system
  • Source Functions: Extend Flink's source abstractions for both streaming and batch
  • Sink Functions: Implement Flink's sink interfaces for data output
  • Rate Limiting: Built-in throttling in TransactionSource for realistic data simulation
  • Sample Data: Predefined transaction dataset with varied amounts for testing fraud detection scenarios

Dependencies

  • Apache Flink: flink-streaming-java_2.12 (provided scope)
  • SLF4J: Logging framework for output formatting
  • Java Standard Library: Core Java APIs for collections, serialization, and SQL timestamp handling

Common Patterns

Fraud Detection Pipeline

import org.apache.flink.streaming.api.windowing.assigners.TumblingProcessingTimeWindows;
import org.apache.flink.streaming.api.windowing.time.Time;

DataStream<Transaction> transactions = env.addSource(new TransactionSource());

DataStream<Alert> alerts = transactions
    .keyBy(Transaction::getAccountId)
    .window(TumblingProcessingTimeWindows.of(Time.minutes(1)))
    .process(new FraudDetectionFunction()) // Custom function
    .map(transaction -> {
        Alert alert = new Alert();
        alert.setId(transaction.getAccountId());
        return alert;
    });

alerts.addSink(new AlertSink());

Batch Transaction Processing

import org.apache.flink.api.java.ExecutionEnvironment;
import org.apache.flink.api.java.DataSet;
import org.apache.flink.types.Row;

ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();

DataSet<Row> transactions = env.createInput(new TransactionRowInputFormat());

DataSet<String> processedData = transactions
    .map(new ProcessTransactionFunction()) // Custom function
    .reduce(new SummarizeFunction()); // Custom function

processedData.output(new LoggerOutputFormat());

Error Handling

The library follows standard Java exception patterns:

  • RuntimeException: Thrown by TransactionSource on thread interruption
  • Standard Java Exceptions: Standard getter/setter and construction exceptions apply
  • Flink Exceptions: All Flink-specific exceptions (serialization, checkpointing) are handled by the framework

docs

index.md

tile.json