0
# Apache Flink Runtime Web
1
2
Apache Flink Runtime Web provides a comprehensive web-based monitoring and management interface for Apache Flink stream processing applications. It combines a Java backend built with Netty for REST API services and an Angular frontend dashboard for real-time visualization and interaction.
3
4
## Package Information
5
6
- **Package Name**: flink-runtime-web
7
- **Package Type**: maven
8
- **Language**: Java (backend) + TypeScript/Angular (frontend)
9
- **Installation**: Include as Maven dependency in Flink applications
10
- **License**: Apache-2.0
11
12
```xml
13
<dependency>
14
<groupId>org.apache.flink</groupId>
15
<artifactId>flink-runtime-web</artifactId>
16
<version>2.1.0</version>
17
</dependency>
18
```
19
20
## Core Imports
21
22
```java
23
// Main components
24
import org.apache.flink.runtime.webmonitor.WebSubmissionExtension;
25
import org.apache.flink.runtime.webmonitor.history.HistoryServer;
26
import org.apache.flink.runtime.webmonitor.utils.WebFrontendBootstrap;
27
28
// JAR management handlers
29
import org.apache.flink.runtime.webmonitor.handlers.JarUploadHandler;
30
import org.apache.flink.runtime.webmonitor.handlers.JarListHandler;
31
import org.apache.flink.runtime.webmonitor.handlers.JarRunHandler;
32
import org.apache.flink.runtime.webmonitor.handlers.JarPlanHandler;
33
import org.apache.flink.runtime.webmonitor.handlers.JarDeleteHandler;
34
35
// Message headers
36
import org.apache.flink.runtime.webmonitor.handlers.JarUploadHeaders;
37
import org.apache.flink.runtime.webmonitor.handlers.JarListHeaders;
38
import org.apache.flink.runtime.webmonitor.handlers.JarRunHeaders;
39
import org.apache.flink.runtime.webmonitor.handlers.JarPlanGetHeaders;
40
import org.apache.flink.runtime.webmonitor.handlers.JarPlanPostHeaders;
41
import org.apache.flink.runtime.webmonitor.handlers.JarDeleteHeaders;
42
43
// Request/Response models
44
import org.apache.flink.runtime.webmonitor.handlers.JarRequestBody;
45
import org.apache.flink.runtime.webmonitor.handlers.JarRunRequestBody;
46
import org.apache.flink.runtime.webmonitor.handlers.JarPlanRequestBody;
47
import org.apache.flink.runtime.webmonitor.handlers.JarUploadResponseBody;
48
import org.apache.flink.runtime.webmonitor.handlers.JarRunResponseBody;
49
import org.apache.flink.runtime.webmonitor.handlers.JarListInfo;
50
51
// Message parameters
52
import org.apache.flink.runtime.webmonitor.handlers.JarMessageParameters;
53
import org.apache.flink.runtime.webmonitor.handlers.JarRunMessageParameters;
54
import org.apache.flink.runtime.webmonitor.handlers.JarPlanMessageParameters;
55
import org.apache.flink.runtime.webmonitor.handlers.JarDeleteMessageParameters;
56
57
// Parameter classes
58
import org.apache.flink.runtime.webmonitor.handlers.JarIdPathParameter;
59
import org.apache.flink.runtime.webmonitor.handlers.EntryClassQueryParameter;
60
import org.apache.flink.runtime.webmonitor.handlers.ParallelismQueryParameter;
61
import org.apache.flink.runtime.webmonitor.handlers.ProgramArgQueryParameter;
62
import org.apache.flink.runtime.webmonitor.handlers.SavepointPathQueryParameter;
63
import org.apache.flink.runtime.webmonitor.handlers.AllowNonRestoredStateQueryParameter;
64
65
// Utilities
66
import org.apache.flink.runtime.webmonitor.handlers.utils.JarHandlerUtils;
67
68
// Core infrastructure
69
import org.apache.flink.runtime.webmonitor.HttpRequestHandler;
70
import org.apache.flink.runtime.webmonitor.PipelineErrorHandler;
71
```
72
73
## Basic Usage
74
75
```java
76
// Start a history server for completed jobs
77
Configuration config = new Configuration();
78
config.setString(HistoryServerOptions.HISTORY_SERVER_WEB_PORT, "8082");
79
config.setString(HistoryServerOptions.HISTORY_SERVER_ARCHIVE_DIRS, "/path/to/archives");
80
81
HistoryServer historyServer = new HistoryServer(config);
82
historyServer.start();
83
84
// Get server port
85
int port = historyServer.getWebPort();
86
System.out.println("History server running on port: " + port);
87
```
88
89
## Architecture
90
91
The Runtime Web module is built around several key components:
92
93
- **Java Backend**: Netty-based HTTP server providing REST APIs for job management, monitoring, and history
94
- **Angular Frontend**: Modern TypeScript dashboard with real-time monitoring, job visualization, and management tools
95
- **JAR Management**: Complete lifecycle management for JAR uploads, execution, and planning
96
- **History Server**: Standalone server for viewing completed job archives
97
- **REST API**: Comprehensive endpoints for programmatic access to all functionality
98
99
## Capabilities
100
101
### JAR Management and Job Submission
102
103
Complete JAR lifecycle management including upload, execution, planning, and deletion. Supports job submission with savepoint restoration and parallelism configuration.
104
105
```java { .api }
106
// JAR Upload Handler
107
public class JarUploadHandler extends AbstractRestHandler<RestfulGateway, EmptyRequestBody, JarUploadResponseBody, EmptyMessageParameters> {
108
public CompletableFuture<JarUploadResponseBody> handleRequest(
109
HandlerRequest<EmptyRequestBody> request,
110
RestfulGateway gateway
111
) throws RestHandlerException;
112
}
113
114
// JAR Run Handler
115
public class JarRunHandler extends AbstractRestHandler<DispatcherGateway, JarRunRequestBody, JarRunResponseBody, JarRunMessageParameters> {
116
public CompletableFuture<JarRunResponseBody> handleRequest(
117
HandlerRequest<JarRunRequestBody> request,
118
DispatcherGateway gateway
119
) throws RestHandlerException;
120
}
121
```
122
123
[JAR Management](./jar-management.md)
124
125
### History Server
126
127
Standalone server providing web interface and REST API for completed job analysis with archive management and static file serving.
128
129
```java { .api }
130
public class HistoryServer {
131
public HistoryServer(Configuration config);
132
public void start() throws Exception;
133
public void stop() throws Exception;
134
public int getWebPort();
135
public static void main(String[] args) throws Exception;
136
}
137
```
138
139
[History Server](./history-server.md)
140
141
### Web Server Infrastructure
142
143
Core HTTP request handling, server bootstrap, and pipeline management using Netty framework.
144
145
```java { .api }
146
public class WebFrontendBootstrap {
147
public WebFrontendBootstrap(
148
Router router,
149
Logger log,
150
File tmpDir,
151
SSLHandlerFactory sslHandlerFactory,
152
String configuredAddress,
153
int configuredPort,
154
Configuration configuration
155
) throws IOException, InterruptedException;
156
157
public int getServerPort();
158
public String getRestAddress();
159
public void shutdown();
160
}
161
```
162
163
[Web Server Infrastructure](./web-server.md)
164
165
### Data Transfer Objects
166
167
Comprehensive request and response models for all REST API operations with proper validation and serialization.
168
169
```java { .api }
170
// JAR Upload Response
171
public class JarUploadResponseBody implements ResponseBody {
172
public JarUploadResponseBody(String filename);
173
public String getFilename();
174
public UploadStatus getStatus();
175
}
176
177
// JAR List Information
178
public class JarListInfo implements ResponseBody {
179
public JarListInfo(String address, List<JarFileInfo> jarFileList);
180
public String getAddress();
181
public List<JarFileInfo> getFiles();
182
183
public static class JarFileInfo {
184
public String getId();
185
public String getName();
186
public long getUploaded();
187
public List<JarEntryInfo> getEntry();
188
}
189
}
190
```
191
192
[Data Transfer Objects](./dto.md)
193
194
### Angular Web Dashboard
195
196
Modern TypeScript dashboard providing real-time monitoring, job visualization, resource management, and developer tools.
197
198
Frontend capabilities include:
199
- **Job Monitoring**: Live job execution graphs, metrics, and performance analysis
200
- **Resource Management**: Task manager and job manager monitoring with detailed metrics
201
- **Visualization**: Interactive DAG visualization using D3/Dagre with flame graphs and charts
202
- **Developer Tools**: Log analysis, thread dumps, profiling, and configuration management
203
204
[Angular Dashboard](./angular-dashboard.md)
205
206
## REST API Endpoints
207
208
| Method | Endpoint | Purpose | Handler Class |
209
|--------|----------|---------|---------------|
210
| POST | `/jars/upload` | Upload JAR files | JarUploadHandler |
211
| GET | `/jars` | List uploaded JARs | JarListHandler |
212
| POST | `/jars/:jarid/run` | Execute JAR file | JarRunHandler |
213
| GET | `/jars/:jarid/plan` | Get execution plan (query params) | JarPlanHandler |
214
| POST | `/jars/:jarid/plan` | Get execution plan (request body) | JarPlanHandler |
215
| DELETE | `/jars/:jarid` | Delete JAR file | JarDeleteHandler |
216
217
### History Server Endpoints
218
| Method | Endpoint | Purpose |
219
|--------|----------|---------|
220
| GET | `/config` | Server configuration |
221
| GET | `/joboverview` | Job overview for history |
222
| GET | `/jobs/:jobid/*` | Historical job details |
223
224
## Configuration
225
226
The module integrates with Flink's configuration system:
227
228
```java { .api }
229
// History Server Options
230
public class HistoryServerOptions {
231
public static final ConfigOption<String> HISTORY_SERVER_WEB_PORT;
232
public static final ConfigOption<String> HISTORY_SERVER_ARCHIVE_DIRS;
233
public static final ConfigOption<Duration> HISTORY_SERVER_ARCHIVE_REFRESH_INTERVAL;
234
}
235
```
236
237
## Error Handling
238
239
The module provides comprehensive error handling through:
240
- `PipelineErrorHandler` for HTTP pipeline errors
241
- REST handler exceptions with proper status codes
242
- Structured error responses in JSON format
243
- Logging integration for debugging and monitoring