0
# History Server
1

2
Standalone server functionality for viewing archived job information and serving static files. The History Server provides long-term job monitoring and analysis capabilities, allowing users to explore completed jobs through a web interface.
3

4
## Capabilities
5

6
### HistoryServer
7

8
Main class for the standalone history server that serves archived job information and web dashboard files.
9

10
```java { .api }
11
/**
12
 * Standalone history server for viewing archived Flink job information.
13
 * Provides web interface for exploring completed jobs and their execution details.
14
 */
15
public class HistoryServer {
16
    /**
17
     * Create a history server with the specified configuration.
18
     * 
19
     * @param configuration Flink configuration containing history server settings
20
     */
21
    public HistoryServer(Configuration configuration);
22
    
23
    /**
24
     * Create a history server with configuration and event listener.
25
     * 
26
     * @param configuration Flink configuration containing history server settings
27
     * @param eventListener Consumer for archive events (job discovery, updates, etc.)
28
     */
29
    public HistoryServer(Configuration configuration, Consumer<ArchiveEvent> eventListener);
30
    
31
    /**
32
     * Main entry point for starting the history server as a standalone application.
33
     * Reads configuration from command line arguments and system properties.
34
     * 
35
     * @param args Command line arguments for server configuration
36
     */
37
    public static void main(String[] args);
38
    
39
    /**
40
     * Start the history server and begin serving requests.
41
     * This method blocks until the server is shut down.
42
     */
43
    public void run();
44
}
45
```
46

47
### HistoryServerArchiveFetcher
48

49
Manages the fetching and processing of job archives for the history server.
50

51
```java { .api }
52
/**
53
 * Fetches and manages job archives for the history server.
54
 * Handles discovery, downloading, and processing of archived job information.
55
 */
56
public class HistoryServerArchiveFetcher {
57
    /**
58
     * Create an archive fetcher with the specified configuration.
59
     * 
60
     * @param refreshDirs List of refresh locations containing paths and file systems
61
     * @param webDir Directory for storing processed web dashboard files
62
     * @param jobArchiveEventListener Consumer for archive events (created/deleted)
63
     * @param cleanupExpiredArchives Whether to clean up expired job archives
64
     * @param maxHistorySize Maximum number of jobs to retain in history (-1 for unlimited)
65
     */
66
    public HistoryServerArchiveFetcher(
67
        List<HistoryServer.RefreshLocation> refreshDirs,
68
        File webDir,
69
        Consumer<ArchiveEvent> jobArchiveEventListener,
70
        boolean cleanupExpiredArchives,
71
        int maxHistorySize
72
    );
73
    
74
    /**
75
     * Fetch archives from all configured directories.
76
     * Scans refresh locations for new job archives and processes them.
77
     */
78
    void fetchArchives();
79
    
80
    /**
81
     * Event representing archive operations in the history server.
82
     */
83
    public static class ArchiveEvent {
84
        public ArchiveEvent(String jobID, ArchiveEventType operation);
85
        public String getJobID();
86
        public ArchiveEventType getType();
87
    }
88
    
89
    /**
90
     * Types of archive events that can occur.
91
     */
92
    public enum ArchiveEventType {
93
        /** Job archive was created in history server. */
94
        CREATED,
95
        /** Job archive was deleted from history server. */
96
        DELETED
97
    }
98
}
99
```
100

101
### HistoryServerStaticFileServerHandler
102

103
Netty handler for serving static files from the web dashboard.
104

105
```java { .api }
106
/**
107
 * Netty channel handler for serving static files for the history server web interface.
108
 * Handles HTTP requests for static assets like HTML, CSS, JavaScript, and images.
109
 */
110
public class HistoryServerStaticFileServerHandler extends SimpleChannelInboundHandler<FullHttpRequest> {
111
    /**
112
     * Create a static file server handler.
113
     * 
114
     * @param webRootDir Root directory containing static web files
115
     */
116
    public HistoryServerStaticFileServerHandler(File webRootDir);
117
    
118
    /**
119
     * Handle incoming HTTP requests for static files.
120
     * Serves files from the configured web root directory.
121
     * 
122
     * @param ctx Netty channel handler context
123
     * @param request HTTP request for static file
124
     */
125
    protected void channelRead0(ChannelHandlerContext ctx, FullHttpRequest request);
126
}
127
```
128

129
## Usage Examples
130

131
### Basic History Server Setup
132

133
```java
134
import org.apache.flink.runtime.webmonitor.history.HistoryServer;
135
import org.apache.flink.configuration.Configuration;
136
import org.apache.flink.configuration.HistoryServerOptions;
137

138
// Configure history server
139
Configuration config = new Configuration();
140
config.setString(HistoryServerOptions.HISTORY_SERVER_WEB_ADDRESS, "0.0.0.0");
141
config.setInteger(HistoryServerOptions.HISTORY_SERVER_WEB_PORT, 8082);
142
config.setString(HistoryServerOptions.HISTORY_SERVER_ARCHIVE_DIRS, "/path/to/archives");
143
config.setLong(HistoryServerOptions.HISTORY_SERVER_ARCHIVE_REFRESH_INTERVAL, 10000L);
144

145
// Start history server
146
HistoryServer historyServer = new HistoryServer(config);
147
historyServer.run(); // Blocks until shutdown
148
```
149

150
### History Server with Event Listening
151

152
```java
153
import org.apache.flink.runtime.webmonitor.history.HistoryServer;
154
import java.util.function.Consumer;
155

156
// Create event listener for archive events
157
Consumer<ArchiveEvent> eventListener = event -> {
158
    switch (event.getType()) {
159
        case ARCHIVE_DISCOVERED:
160
            System.out.println("New archive discovered: " + event.getArchivePath());
161
            break;
162
        case ARCHIVE_UPDATED:
163
            System.out.println("Archive updated: " + event.getArchivePath());
164
            break;
165
        case ARCHIVE_REMOVED:
166
            System.out.println("Archive removed: " + event.getArchivePath());
167
            break;
168
    }
169
};
170

171
// Create history server with event listener
172
HistoryServer historyServer = new HistoryServer(config, eventListener);
173
historyServer.run();
174
```
175

176
### Standalone History Server Application
177

178
```java
179
// Run as standalone application
180
public class HistoryServerApplication {
181
    public static void main(String[] args) {
182
        // Pass configuration through command line arguments
183
        // Example: --historyserver.web.address=0.0.0.0 --historyserver.web.port=8082
184
        HistoryServer.main(args);
185
    }
186
}
187
```
188

189
### Custom Archive Fetcher Configuration
190

191
```java
192
import org.apache.flink.runtime.webmonitor.history.HistoryServerArchiveFetcher;
193
import java.io.File;
194
import java.util.Arrays;
195
import java.util.List;
196

197
// Configure archive directories
198
List<HistoryServerArchiveFetcher.ArchiveDirectory> archiveDirs = Arrays.asList(
199
    new HistoryServerArchiveFetcher.ArchiveDirectory("/path/to/local/archives"),
200
    new HistoryServerArchiveFetcher.ArchiveDirectory("hdfs://namenode:port/flink-archives"),
201
    new HistoryServerArchiveFetcher.ArchiveDirectory("s3://bucket/flink-archives")
202
);
203

204
// Create archive fetcher
205
HistoryServerArchiveFetcher fetcher = new HistoryServerArchiveFetcher(
206
    10000L,                    // Refresh every 10 seconds
207
    archiveDirs,              // Archive directories to scan
208
    new File("/web/dashboard"), // Web dashboard files
209
    eventListener             // Event listener for archive changes
210
);
211

212
// Start fetcher
213
fetcher.start();
214

215
// Stop when done
216
fetcher.stop();
217
```
218

219
### Static File Server Integration
220

221
```java
222
import org.apache.flink.runtime.webmonitor.history.HistoryServerStaticFileServerHandler;
223
import org.apache.flink.shaded.netty4.io.netty.channel.ChannelPipeline;
224
import java.io.File;
225

226
// Setup static file serving in Netty pipeline
227
public void initChannel(SocketChannel ch) {
228
    ChannelPipeline pipeline = ch.pipeline();
229
    
230
    // Add static file handler for web dashboard assets
231
    File webRoot = new File("/path/to/web-dashboard");
232
    HistoryServerStaticFileServerHandler staticHandler = 
233
        new HistoryServerStaticFileServerHandler(webRoot);
234
    
235
    pipeline.addLast("staticFileHandler", staticHandler);
236
}
237
```
238

239
## Configuration Options
240

241
The History Server supports extensive configuration through Flink's configuration system:
242

243
```java
244
// Common configuration options
245
Configuration config = new Configuration();
246

247
// Server binding
248
config.setString(HistoryServerOptions.HISTORY_SERVER_WEB_ADDRESS, "localhost");
249
config.setInteger(HistoryServerOptions.HISTORY_SERVER_WEB_PORT, 8082);
250

251
// Archive locations
252
config.setString(HistoryServerOptions.HISTORY_SERVER_ARCHIVE_DIRS, 
253
    "/local/archives,hdfs://namenode/archives");
254

255
// Refresh settings
256
config.setLong(HistoryServerOptions.HISTORY_SERVER_ARCHIVE_REFRESH_INTERVAL, 10000L);
257

258
// Web dashboard location
259
config.setString(HistoryServerOptions.HISTORY_SERVER_WEB_DIR, "/path/to/web-dashboard");
260

261
// SSL configuration (optional)
262
config.setBoolean(SecurityOptions.SSL_ENABLED, true);
263
config.setString(SecurityOptions.SSL_KEYSTORE, "/path/to/keystore");
264
config.setString(SecurityOptions.SSL_KEYSTORE_PASSWORD, "password");
265
```
266

267
## Archive Event Types
268

269
The History Server supports event-driven architecture for archive management:
270

271
```java
272
// Archive event types available to event listeners
273
public enum ArchiveEventType {
274
    ARCHIVE_DISCOVERED,  // New archive found
275
    ARCHIVE_UPDATED,     // Existing archive updated
276
    ARCHIVE_REMOVED      // Archive no longer available
277
}
278

279
// Event handling example
280
Consumer<ArchiveEvent> listener = event -> {
281
    String archivePath = event.getArchivePath();
282
    JobID jobId = event.getJobId();
283
    ArchiveEventType type = event.getType();
284
    
285
    // Custom handling based on event type
286
    handleArchiveEvent(type, archivePath, jobId);
287
};
288
```
289

290
## Error Handling
291

292
The History Server includes robust error handling for common scenarios:
293

294
- **Archive access failures**: When archive directories are inaccessible
295
- **Corrupt archives**: When job archive files are damaged or invalid
296
- **Network issues**: When accessing remote archive locations (HDFS, S3, etc.)
297
- **Web server failures**: When the HTTP server cannot start or bind to port
298
- **Resource cleanup**: Proper cleanup when shutting down the server