0
# Flink Metrics JMX
1

2
Flink Metrics JMX is a metrics reporter implementation for Apache Flink that exposes Flink's internal metrics (counters, gauges, histograms, and meters) through Java Management Extensions (JMX). It enables real-time monitoring and management of Flink streaming and batch processing applications through standard JMX interfaces, making it essential for production deployments requiring comprehensive observability and operational insights.
3

4
## Package Information
5

6
- **Group ID**: org.apache.flink
7
- **Artifact ID**: flink-metrics-jmx
8
- **Package Type**: maven
9
- **Language**: Java
10
- **Version**: 2.1.0
11
- **License**: Apache-2.0
12
- **Installation**: Include as dependency in your Maven/Gradle project alongside Flink
13

14
**Maven Dependency:**
15

16
```xml
17
<dependency>
18
    <groupId>org.apache.flink</groupId>
19
    <artifactId>flink-metrics-jmx</artifactId>
20
    <version>2.1.0</version>
21
</dependency>
22
```
23

24
**Gradle Dependency:**
25

26
```groovy
27
implementation 'org.apache.flink:flink-metrics-jmx:2.1.0'
28
```
29

30
## Core Imports
31

32
```java
33
import org.apache.flink.metrics.jmx.JMXReporter;
34
import org.apache.flink.metrics.jmx.JMXReporterFactory;
35
// MBean interfaces are inner interfaces of JMXReporter:
36
// JMXReporter.JmxCounterMBean, JMXReporter.JmxGaugeMBean, etc.
37
```
38

39
## Basic Usage
40

41
The JMX metrics reporter is typically configured through Flink's configuration system and operates automatically:
42

43
```java
44
// The reporter is usually instantiated by Flink's metrics system
45
// via the factory pattern using configuration properties
46
Properties config = new Properties();
47
config.setProperty("port", "9999"); // Optional: JMX server port
48

49
JMXReporterFactory factory = new JMXReporterFactory();
50
JMXReporter reporter = factory.createMetricReporter(config);
51

52
// Reporter lifecycle is managed by Flink
53
reporter.open(new MetricConfig());
54

55
// Check if JMX server port is available
56
Optional<Integer> port = reporter.getPort();
57
if (port.isPresent()) {
58
    System.out.println("JMX server available on port: " + port.get());
59
}
60
```
61

62
**Configuration in Flink:**
63

64
```yaml
65
metrics:
66
  reporters:
67
    - class: org.apache.flink.metrics.jmx.JMXReporterFactory
68
      port: 9999  # Optional JMX server port
69
```
70

71
## Architecture
72

73
The JMX metrics reporter is built around several key components:
74

75
- **JMXReporter**: Main reporter class that implements the MetricReporter interface and manages JMX MBean registration/deregistration
76
- **JMXReporterFactory**: Factory class implementing MetricReporterFactory for service provider interface integration
77
- **JMX MBean Interfaces**: Strongly-typed interfaces for different metric types (Counter, Gauge, Histogram, Meter)
78
- **MBean Server Integration**: Uses Java's platform MBeanServer for JMX registration
79
- **Service Provider Interface**: Registered via META-INF/services for automatic discovery by Flink
80

81
## Capabilities
82

83
### Metric Reporter Factory
84

85
Factory class for creating JMXReporter instances, implementing the standard Flink MetricReporterFactory interface.
86

87
```java { .api }
88
/**
89
 * Factory for creating JMXReporter instances
90
 */
91
public class JMXReporterFactory implements MetricReporterFactory {
92
    /** Configuration key for port setting */
93
    public static final String ARG_PORT = "port";
94
    
95
    /**
96
     * Creates a new JMXReporter instance with the given configuration
97
     * @param properties Configuration properties for the reporter
98
     * @return Configured JMXReporter instance
99
     */
100
    public JMXReporter createMetricReporter(Properties properties);
101
}
102
```
103

104
### JMX Metrics Reporter
105

106
Core reporter class that exports Flink metrics as JMX MBeans, with automatic registration and deregistration capabilities. The reporter is thread-safe and handles concurrent metric registration.
107

108
```java { .api }
109
/**
110
 * MetricReporter that exports Metrics via JMX
111
 */
112
public class JMXReporter implements MetricReporter {
113
    /** JMX domain prefix for all Flink metrics */
114
    public static final String JMX_DOMAIN_PREFIX = "org.apache.flink.";
115
    
116
    /**
117
     * Opens the reporter with the given configuration
118
     * @param config Metric configuration
119
     */
120
    public void open(MetricConfig config);
121
    
122
    /**
123
     * Closes the reporter and cleans up resources
124
     */
125
    public void close();
126
    
127
    /**
128
     * Gets the port of the JMX server if available
129
     * @return Optional containing the port number, empty if not available
130
     */
131
    public Optional<Integer> getPort();
132
    
133
    /**
134
     * Called when a metric is added to register it as a JMX MBean
135
     * @param metric The metric to register
136
     * @param metricName Name of the metric
137
     * @param group Metric group containing variables and scope information
138
     */
139
    public void notifyOfAddedMetric(Metric metric, String metricName, MetricGroup group);
140
    
141
    /**
142
     * Called when a metric is removed to unregister it from JMX
143
     * @param metric The metric to unregister
144
     * @param metricName Name of the metric
145
     * @param group Metric group containing variables and scope information
146
     */
147
    public void notifyOfRemovedMetric(Metric metric, String metricName, MetricGroup group);
148
}
149
```
150

151
### JMX Domain and Name Generation
152

153
Static utility methods for generating JMX-compliant domain names and property tables.
154

155
```java { .api }
156
/**
157
 * Generates JMX property table from metric group variables
158
 * @param variables Map of variable names to values
159
 * @return Hashtable suitable for JMX ObjectName construction
160
 */
161
public static Hashtable<String, String> generateJmxTable(Map<String, String> variables);
162

163
/**
164
 * Generates JMX domain name for a metric
165
 * @param metricName Name of the metric
166
 * @param group Metric group providing logical scope
167
 * @return Complete JMX domain name
168
 */
169
public static String generateJmxDomain(String metricName, MetricGroup group);
170

171
/**
172
 * Replaces characters invalid for JMX names with valid alternatives
173
 * @param str String to process
174
 * @return String with invalid characters replaced
175
 */
176
public static String replaceInvalidChars(String str);
177
```
178

179
### JMX MBean Interfaces
180

181
Standard JMX MBean interfaces for different metric types, enabling type-safe access to metric values. All interfaces are inner interfaces of the JMXReporter class.
182

183
```java { .api }
184
/**
185
 * Base interface for all JMX metric beans
186
 */
187
public interface MetricMBean {
188
}
189

190
/**
191
 * JMX interface for counter metrics
192
 */
193
public interface JmxCounterMBean extends MetricMBean {
194
    /**
195
     * Gets the current count value
196
     * @return Current count
197
     */
198
    long getCount();
199
}
200

201
/**
202
 * JMX interface for gauge metrics
203
 */
204
public interface JmxGaugeMBean extends MetricMBean {
205
    /**
206
     * Gets the current gauge value
207
     * @return Current gauge value
208
     */
209
    Object getValue();
210
}
211

212
/**
213
 * JMX interface for histogram metrics providing statistical information
214
 */
215
public interface JmxHistogramMBean extends MetricMBean {
216
    /** Gets the number of recorded values */
217
    long getCount();
218
    
219
    /** Gets the arithmetic mean of recorded values */
220
    double getMean();
221
    
222
    /** Gets the standard deviation of recorded values */
223
    double getStdDev();
224
    
225
    /** Gets the maximum recorded value */
226
    long getMax();
227
    
228
    /** Gets the minimum recorded value */
229
    long getMin();
230
    
231
    /** Gets the median (50th percentile) */
232
    double getMedian();
233
    
234
    /** Gets the 75th percentile */
235
    double get75thPercentile();
236
    
237
    /** Gets the 95th percentile */
238
    double get95thPercentile();
239
    
240
    /** Gets the 98th percentile */
241
    double get98thPercentile();
242
    
243
    /** Gets the 99th percentile */
244
    double get99thPercentile();
245
    
246
    /** Gets the 99.9th percentile */
247
    double get999thPercentile();
248
}
249

250
/**
251
 * JMX interface for meter metrics tracking rates and counts
252
 */
253
public interface JmxMeterMBean extends MetricMBean {
254
    /**
255
     * Gets the current rate
256
     * @return Current rate value
257
     */
258
    double getRate();
259
    
260
    /**
261
     * Gets the total count
262
     * @return Total count value
263
     */
264
    long getCount();
265
}
266
```
267

268
## Configuration
269

270
### Reporter Configuration
271

272
The JMX reporter supports the following configuration options:
273

274
- **port**: (Optional) JMX server port configuration. Can specify a single port (e.g., "9999") or a port range (e.g., "9000-9010"). This option is deprecated; use `JMXServerOptions.JMX_SERVER_PORT` instead.
275

276
### JMX Domain Structure
277

278
Metrics are registered under the JMX domain following this pattern:
279
```
280
org.apache.flink.<logical_scope>.<metric_name>
281
```
282

283
For example:
284
- Counter: `org.apache.flink.jobmanager.numRunningJobs`
285
- Gauge: `org.apache.flink.taskmanager.heap.used`
286
- Histogram: `org.apache.flink.job.task.buffers.inputQueueLength`
287

288
### Character Replacement
289

290
Invalid JMX characters are automatically replaced:
291
- Quotes (`"`, `>`, `<`) are removed
292
- Spaces are replaced with underscores (`_`)
293
- Special characters (`,`, `=`, `;`, `:`, `?`, `'`, `*`) are replaced with hyphens (`-`)
294

295
## Usage Examples
296

297
### Programmatic Reporter Creation
298

299
```java
300
import org.apache.flink.metrics.jmx.JMXReporterFactory;
301
import java.util.Properties;
302

303
// Create reporter factory
304
JMXReporterFactory factory = new JMXReporterFactory();
305

306
// Configure properties
307
Properties properties = new Properties();
308
properties.setProperty(JMXReporterFactory.ARG_PORT, "9999");
309

310
// Create reporter instance
311
JMXReporter reporter = factory.createMetricReporter(properties);
312

313
// Open reporter (normally done by Flink)
314
reporter.open(new MetricConfig());
315

316
// Check for JMX server availability
317
reporter.getPort().ifPresent(port -> 
318
    System.out.println("JMX metrics available on port: " + port)
319
);
320
```
321

322
### JMX Client Connection
323

324
```java
325
import javax.management.remote.JMXConnector;
326
import javax.management.remote.JMXConnectorFactory;
327
import javax.management.remote.JMXServiceURL;
328
import javax.management.MBeanServerConnection;
329
import javax.management.ObjectName;
330

331
// Connect to JMX server
332
JMXServiceURL url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmi");
333
JMXConnector connector = JMXConnectorFactory.connect(url);
334
MBeanServerConnection connection = connector.getMBeanServerConnection();
335

336
// Query Flink metrics
337
ObjectName pattern = new ObjectName("org.apache.flink.*:*");
338
Set<ObjectName> metrics = connection.queryNames(pattern, null);
339

340
for (ObjectName metric : metrics) {
341
    System.out.println("Metric: " + metric);
342
    // Access metric values through the connection
343
}
344

345
connector.close();
346
```
347

348
### Configuration in Flink Configuration
349

350
```yaml
351
# conf/flink-conf.yaml
352
metrics.reporters: jmx
353
metrics.reporter.jmx.factory.class: org.apache.flink.metrics.jmx.JMXReporterFactory
354
metrics.reporter.jmx.port: 8789
355
```