or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

angular-dashboard.mddto.mdhistory-server.mdindex.mdjar-management.mdweb-server.md
tile.json

tessl/maven-org-apache-flink--flink-runtime-web

Web-based monitoring and management interface for Apache Flink with REST APIs and Angular dashboard.

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
mavenpkg:maven/org.apache.flink/flink-runtime-web@2.1.x

To install, run

npx @tessl/cli install tessl/maven-org-apache-flink--flink-runtime-web@2.1.0

index.mddocs/

Apache Flink Runtime Web

Apache Flink Runtime Web provides a comprehensive web-based monitoring and management interface for Apache Flink stream processing applications. It combines a Java backend built with Netty for REST API services and an Angular frontend dashboard for real-time visualization and interaction.

Package Information

  • Package Name: flink-runtime-web
  • Package Type: maven
  • Language: Java (backend) + TypeScript/Angular (frontend)
  • Installation: Include as Maven dependency in Flink applications
  • License: Apache-2.0
<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-runtime-web</artifactId>
    <version>2.1.0</version>
</dependency>

Core Imports

// Main components
import org.apache.flink.runtime.webmonitor.WebSubmissionExtension;
import org.apache.flink.runtime.webmonitor.history.HistoryServer;
import org.apache.flink.runtime.webmonitor.utils.WebFrontendBootstrap;

// JAR management handlers
import org.apache.flink.runtime.webmonitor.handlers.JarUploadHandler;
import org.apache.flink.runtime.webmonitor.handlers.JarListHandler;
import org.apache.flink.runtime.webmonitor.handlers.JarRunHandler;
import org.apache.flink.runtime.webmonitor.handlers.JarPlanHandler;
import org.apache.flink.runtime.webmonitor.handlers.JarDeleteHandler;

// Message headers
import org.apache.flink.runtime.webmonitor.handlers.JarUploadHeaders;
import org.apache.flink.runtime.webmonitor.handlers.JarListHeaders;
import org.apache.flink.runtime.webmonitor.handlers.JarRunHeaders;
import org.apache.flink.runtime.webmonitor.handlers.JarPlanGetHeaders;
import org.apache.flink.runtime.webmonitor.handlers.JarPlanPostHeaders;
import org.apache.flink.runtime.webmonitor.handlers.JarDeleteHeaders;

// Request/Response models
import org.apache.flink.runtime.webmonitor.handlers.JarRequestBody;
import org.apache.flink.runtime.webmonitor.handlers.JarRunRequestBody;
import org.apache.flink.runtime.webmonitor.handlers.JarPlanRequestBody;
import org.apache.flink.runtime.webmonitor.handlers.JarUploadResponseBody;
import org.apache.flink.runtime.webmonitor.handlers.JarRunResponseBody;
import org.apache.flink.runtime.webmonitor.handlers.JarListInfo;

// Message parameters
import org.apache.flink.runtime.webmonitor.handlers.JarMessageParameters;
import org.apache.flink.runtime.webmonitor.handlers.JarRunMessageParameters;
import org.apache.flink.runtime.webmonitor.handlers.JarPlanMessageParameters;
import org.apache.flink.runtime.webmonitor.handlers.JarDeleteMessageParameters;

// Parameter classes
import org.apache.flink.runtime.webmonitor.handlers.JarIdPathParameter;
import org.apache.flink.runtime.webmonitor.handlers.EntryClassQueryParameter;
import org.apache.flink.runtime.webmonitor.handlers.ParallelismQueryParameter;
import org.apache.flink.runtime.webmonitor.handlers.ProgramArgQueryParameter;
import org.apache.flink.runtime.webmonitor.handlers.SavepointPathQueryParameter;
import org.apache.flink.runtime.webmonitor.handlers.AllowNonRestoredStateQueryParameter;

// Utilities
import org.apache.flink.runtime.webmonitor.handlers.utils.JarHandlerUtils;

// Core infrastructure
import org.apache.flink.runtime.webmonitor.HttpRequestHandler;
import org.apache.flink.runtime.webmonitor.PipelineErrorHandler;

Basic Usage

// Start a history server for completed jobs
Configuration config = new Configuration();
config.setString(HistoryServerOptions.HISTORY_SERVER_WEB_PORT, "8082");
config.setString(HistoryServerOptions.HISTORY_SERVER_ARCHIVE_DIRS, "/path/to/archives");

HistoryServer historyServer = new HistoryServer(config);
historyServer.start();

// Get server port
int port = historyServer.getWebPort();
System.out.println("History server running on port: " + port);

Architecture

The Runtime Web module is built around several key components:

  • Java Backend: Netty-based HTTP server providing REST APIs for job management, monitoring, and history
  • Angular Frontend: Modern TypeScript dashboard with real-time monitoring, job visualization, and management tools
  • JAR Management: Complete lifecycle management for JAR uploads, execution, and planning
  • History Server: Standalone server for viewing completed job archives
  • REST API: Comprehensive endpoints for programmatic access to all functionality

Capabilities

JAR Management and Job Submission

Complete JAR lifecycle management including upload, execution, planning, and deletion. Supports job submission with savepoint restoration and parallelism configuration.

// JAR Upload Handler
public class JarUploadHandler extends AbstractRestHandler<RestfulGateway, EmptyRequestBody, JarUploadResponseBody, EmptyMessageParameters> {
    public CompletableFuture<JarUploadResponseBody> handleRequest(
        HandlerRequest<EmptyRequestBody> request, 
        RestfulGateway gateway
    ) throws RestHandlerException;
}

// JAR Run Handler  
public class JarRunHandler extends AbstractRestHandler<DispatcherGateway, JarRunRequestBody, JarRunResponseBody, JarRunMessageParameters> {
    public CompletableFuture<JarRunResponseBody> handleRequest(
        HandlerRequest<JarRunRequestBody> request,
        DispatcherGateway gateway
    ) throws RestHandlerException;
}

JAR Management

History Server

Standalone server providing web interface and REST API for completed job analysis with archive management and static file serving.

public class HistoryServer {
    public HistoryServer(Configuration config);
    public void start() throws Exception;
    public void stop() throws Exception;
    public int getWebPort();
    public static void main(String[] args) throws Exception;
}

History Server

Web Server Infrastructure

Core HTTP request handling, server bootstrap, and pipeline management using Netty framework.

public class WebFrontendBootstrap {
    public WebFrontendBootstrap(
        Router router,
        Logger log, 
        File tmpDir,
        SSLHandlerFactory sslHandlerFactory,
        String configuredAddress,
        int configuredPort,
        Configuration configuration
    ) throws IOException, InterruptedException;
    
    public int getServerPort();
    public String getRestAddress();
    public void shutdown();
}

Web Server Infrastructure

Data Transfer Objects

Comprehensive request and response models for all REST API operations with proper validation and serialization.

// JAR Upload Response
public class JarUploadResponseBody implements ResponseBody {
    public JarUploadResponseBody(String filename);
    public String getFilename();
    public UploadStatus getStatus();
}

// JAR List Information
public class JarListInfo implements ResponseBody {
    public JarListInfo(String address, List<JarFileInfo> jarFileList);
    public String getAddress();
    public List<JarFileInfo> getFiles();
    
    public static class JarFileInfo {
        public String getId();
        public String getName();
        public long getUploaded();
        public List<JarEntryInfo> getEntry();
    }
}

Data Transfer Objects

Angular Web Dashboard

Modern TypeScript dashboard providing real-time monitoring, job visualization, resource management, and developer tools.

Frontend capabilities include:

  • Job Monitoring: Live job execution graphs, metrics, and performance analysis
  • Resource Management: Task manager and job manager monitoring with detailed metrics
  • Visualization: Interactive DAG visualization using D3/Dagre with flame graphs and charts
  • Developer Tools: Log analysis, thread dumps, profiling, and configuration management

Angular Dashboard

REST API Endpoints

MethodEndpointPurposeHandler Class
POST/jars/uploadUpload JAR filesJarUploadHandler
GET/jarsList uploaded JARsJarListHandler
POST/jars/:jarid/runExecute JAR fileJarRunHandler
GET/jars/:jarid/planGet execution plan (query params)JarPlanHandler
POST/jars/:jarid/planGet execution plan (request body)JarPlanHandler
DELETE/jars/:jaridDelete JAR fileJarDeleteHandler

History Server Endpoints

MethodEndpointPurpose
GET/configServer configuration
GET/joboverviewJob overview for history
GET/jobs/:jobid/*Historical job details

Configuration

The module integrates with Flink's configuration system:

// History Server Options
public class HistoryServerOptions {
    public static final ConfigOption<String> HISTORY_SERVER_WEB_PORT;
    public static final ConfigOption<String> HISTORY_SERVER_ARCHIVE_DIRS;
    public static final ConfigOption<Duration> HISTORY_SERVER_ARCHIVE_REFRESH_INTERVAL;
}

Error Handling

The module provides comprehensive error handling through:

  • PipelineErrorHandler for HTTP pipeline errors
  • REST handler exceptions with proper status codes
  • Structured error responses in JSON format
  • Logging integration for debugging and monitoring