0
# Metric Organization
1

2
Hierarchical metric organization system for creating logical namespaces and managing metric lifecycles. MetricGroup provides the primary interface for registering metrics and creating nested organizational structures.
3

4
## Capabilities
5

6
### MetricGroup Interface
7

8
Core interface for organizing metrics into hierarchical namespaces. Supports metric registration, group creation, and scope management.
9

10
```java { .api }
11
/**
12
 * A MetricGroup is a named container for Metrics and further metric subgroups.
13
 * Instances of this class can be used to register new metrics with Flink and 
14
 * to create a nested hierarchy based on the group names.
15
 */
16
public interface MetricGroup {
17
    
18
    // Counter registration methods
19
    /**
20
     * Creates and registers a new Counter with Flink.
21
     * @param name name of the counter
22
     * @return the created counter
23
     */
24
    Counter counter(String name);
25
    
26
    /**
27
     * Creates and registers a new Counter with Flink.
28
     * @param name name of the counter (as integer)
29
     * @return the created counter
30
     */
31
    default Counter counter(int name);
32
    
33
    /**
34
     * Registers a Counter with Flink.
35
     * @param name name of the counter
36
     * @param counter counter to register
37
     * @return the given counter
38
     */
39
    <C extends Counter> C counter(String name, C counter);
40
    
41
    /**
42
     * Registers a Counter with Flink.
43
     * @param name name of the counter (as integer)
44
     * @param counter counter to register
45
     * @return the given counter
46
     */
47
    default <C extends Counter> C counter(int name, C counter);
48
    
49
    // Gauge registration methods
50
    /**
51
     * Registers a new Gauge with Flink.
52
     * @param name name of the gauge
53
     * @param gauge gauge to register
54
     * @return the given gauge
55
     */
56
    <T, G extends Gauge<T>> G gauge(String name, G gauge);
57
    
58
    /**
59
     * Registers a new Gauge with Flink.
60
     * @param name name of the gauge (as integer)
61
     * @param gauge gauge to register
62
     * @return the given gauge
63
     */
64
    default <T, G extends Gauge<T>> G gauge(int name, G gauge);
65
    
66
    // Histogram registration methods
67
    /**
68
     * Registers a new Histogram with Flink.
69
     * @param name name of the histogram
70
     * @param histogram histogram to register
71
     * @return the registered histogram
72
     */
73
    <H extends Histogram> H histogram(String name, H histogram);
74
    
75
    /**
76
     * Registers a new Histogram with Flink.
77
     * @param name name of the histogram (as integer)
78
     * @param histogram histogram to register
79
     * @return the registered histogram
80
     */
81
    default <H extends Histogram> H histogram(int name, H histogram);
82
    
83
    // Meter registration methods
84
    /**
85
     * Registers a new Meter with Flink.
86
     * @param name name of the meter
87
     * @param meter meter to register
88
     * @return the registered meter
89
     */
90
    <M extends Meter> M meter(String name, M meter);
91
    
92
    /**
93
     * Registers a new Meter with Flink.
94
     * @param name name of the meter (as integer)
95
     * @param meter meter to register
96
     * @return the registered meter
97
     */
98
    default <M extends Meter> M meter(int name, M meter);
99
    
100
    // Group creation methods
101
    /**
102
     * Creates a new MetricGroup and adds it to this groups sub-groups.
103
     * @param name name of the group
104
     * @return the created group
105
     */
106
    MetricGroup addGroup(String name);
107
    
108
    /**
109
     * Creates a new MetricGroup and adds it to this groups sub-groups.
110
     * @param name name of the group (as integer)
111
     * @return the created group
112
     */
113
    default MetricGroup addGroup(int name);
114
    
115
    /**
116
     * Creates a new key-value MetricGroup pair. The key group is added to 
117
     * this groups sub-groups, while the value group is added to the key 
118
     * group's sub-groups. This method returns the value group.
119
     * @param key name of the first group
120
     * @param value name of the second group
121
     * @return the second created group
122
     */
123
    MetricGroup addGroup(String key, String value);
124
    
125
    // Scope and identification methods
126
    /**
127
     * Gets the scope as an array of the scope components.
128
     * @return scope components array
129
     */
130
    String[] getScopeComponents();
131
    
132
    /**
133
     * Returns a map of all variables and their associated value.
134
     * @return map of all variables and their associated value
135
     */
136
    Map<String, String> getAllVariables();
137
    
138
    /**
139
     * Returns the fully qualified metric name.
140
     * @param metricName metric name
141
     * @return fully qualified metric name
142
     */
143
    String getMetricIdentifier(String metricName);
144
    
145
    /**
146
     * Returns the fully qualified metric name with character filtering.
147
     * @param metricName metric name
148
     * @param filter character filter applied to scope components
149
     * @return fully qualified metric name
150
     */
151
    String getMetricIdentifier(String metricName, CharacterFilter filter);
152
    
153
    // Experimental span support
154
    /**
155
     * Adds a span to this metric group (experimental feature).
156
     * @param spanBuilder span builder to add
157
     */
158
    @Experimental
159
    default void addSpan(SpanBuilder spanBuilder) {}
160
}
161
```
162

163
**Usage Examples:**
164

165
```java
166
// Basic metric registration
167
MetricGroup rootGroup = // ... obtained from runtime context
168
Counter totalEvents = rootGroup.counter("total-events");
169
Gauge<Integer> queueSize = rootGroup.gauge("queue-size", () -> queue.size());
170

171
// Hierarchical organization
172
MetricGroup operatorGroup = rootGroup.addGroup("operators");
173
MetricGroup mapOpGroup = operatorGroup.addGroup("map-operator");
174
Counter mapRecords = mapOpGroup.counter("records-processed");
175

176
// Key-value groups for dynamic naming
177
MetricGroup taskGroup = rootGroup.addGroup("task", taskId);
178
Counter taskCounter = taskGroup.counter("events");
179

180
// Multiple nesting levels
181
MetricGroup deepGroup = rootGroup
182
    .addGroup("processing")
183
    .addGroup("stage-1")  
184
    .addGroup("partition", String.valueOf(partitionId));
185

186
// Using integer names for dynamic metrics
187
for (int i = 0; i < numPartitions; i++) {
188
    MetricGroup partitionGroup = rootGroup.addGroup("partition", String.valueOf(i));
189
    Counter partitionCounter = partitionGroup.counter("records");
190
}
191
```
192

193
### Logical Scope Provider
194

195
Extension interface for metric groups that support logical scopes, providing additional scope formatting capabilities.
196

197
```java { .api }
198
/**
199
 * Extension for metric groups that support logical scopes.
200
 * This interface removes the need for reporters to depend on flink-runtime 
201
 * to access the logical scope.
202
 */
203
public interface LogicalScopeProvider {
204
    /**
205
     * Returns the logical scope for the metric group with the given filter 
206
     * applied to all scope components.
207
     * @param filter filter to apply to all scope components
208
     * @return logical scope
209
     */
210
    String getLogicalScope(CharacterFilter filter);
211
    
212
    /**
213
     * Returns the logical scope for the metric group with the given filter 
214
     * applied and the given delimiter used to concatenate scope components.
215
     * @param filter filter to apply to all scope components
216
     * @param delimiter delimiter to use for concatenating scope components
217
     * @return logical scope
218
     */
219
    String getLogicalScope(CharacterFilter filter, char delimiter);
220
    
221
    /**
222
     * Returns the underlying metric group.
223
     * @return underlying metric group
224
     */
225
    MetricGroup getWrappedMetricGroup();
226
    
227
    /**
228
     * Casts the given metric group to a LogicalScopeProvider, if it 
229
     * implements the interface.
230
     * @param metricGroup metric group to cast
231
     * @return cast metric group
232
     * @throws IllegalStateException if the metric group did not implement 
233
     *         the LogicalScopeProvider interface
234
     */
235
    static LogicalScopeProvider castFrom(MetricGroup metricGroup) 
236
            throws IllegalStateException {
237
        if (metricGroup instanceof LogicalScopeProvider) {
238
            return (LogicalScopeProvider) metricGroup;
239
        } else {
240
            throw new IllegalStateException(
241
                "The given metric group does not implement the LogicalScopeProvider interface.");
242
        }
243
    }
244
}
245
```
246

247
**Usage Examples:**
248

249
```java
250
// Check if group supports logical scopes
251
if (metricGroup instanceof LogicalScopeProvider) {
252
    LogicalScopeProvider provider = (LogicalScopeProvider) metricGroup;
253
    String logicalScope = provider.getLogicalScope(CharacterFilter.NO_OP_FILTER);
254
}
255

256
// Safe casting with error handling
257
try {
258
    LogicalScopeProvider provider = LogicalScopeProvider.castFrom(metricGroup);
259
    String scope = provider.getLogicalScope(characterFilter, '.');
260
} catch (IllegalStateException e) {
261
    // Handle groups that don't support logical scopes
262
}
263

264
// Custom delimiter usage
265
LogicalScopeProvider provider = LogicalScopeProvider.castFrom(metricGroup);
266
String underscoreScope = provider.getLogicalScope(filter, '_');
267
String colonScope = provider.getLogicalScope(filter, ':');
268
```
269

270
### Scope Management
271

272
Understanding how metric scopes work for building qualified metric names.
273

274
**Scope Components:**
275

276
```java
277
// Example scope hierarchy: root -> operators -> map-op -> subtask-0
278
MetricGroup rootGroup = getRootGroup();           // []
279
MetricGroup opGroup = rootGroup.addGroup("operators");      // ["operators"]
280
MetricGroup mapGroup = opGroup.addGroup("map-op");          // ["operators", "map-op"]  
281
MetricGroup subtaskGroup = mapGroup.addGroup("subtask-0");  // ["operators", "map-op", "subtask-0"]
282

283
// Get scope information
284
String[] components = subtaskGroup.getScopeComponents();
285
// Result: ["operators", "map-op", "subtask-0"]
286

287
String identifier = subtaskGroup.getMetricIdentifier("records-processed");
288
// Result: "operators.map-op.subtask-0.records-processed"
289
```
290

291
**Variable Substitution:**
292

293
```java
294
// Key-value groups create variables
295
MetricGroup taskGroup = rootGroup.addGroup("task", "task-123");
296
Map<String, String> variables = taskGroup.getAllVariables();
297
// Result: {"task": "task-123"}
298

299
// Variables can be used in scope templates by reporters
300
String template = "<host>.<tm_id>.<job_name>.<task>.<metric_name>";
301
// Variables would be substituted during reporting
302
```
303

304
**Character Filtering:**
305

306
```java
307
// Define custom character filter
308
CharacterFilter dotToUnderscore = input -> input.replace('.', '_');
309

310
// Apply filter to metric identifier
311
String filtered = metricGroup.getMetricIdentifier("my.metric.name", dotToUnderscore);
312
// Transforms scope components but not the metric name itself
313
```
314

315
### Unregistered Metrics Group
316

317
No-operation implementation that doesn't register metrics with the metrics system, useful for testing or disabled metrics scenarios.
318

319
```java { .api }
320
/**
321
 * A special MetricGroup that does not register any metrics at the 
322
 * metrics registry and any reporters.
323
 */
324
@Internal
325
public class UnregisteredMetricsGroup implements MetricGroup {
326
    // All methods return no-op implementations or the passed-in metrics
327
    // without actual registration
328
}
329
```
330

331
**Usage Examples:**
332

333
```java
334
// Create unregistered group for testing
335
MetricGroup testGroup = new UnregisteredMetricsGroup();
336

337
// Metrics are created but not reported
338
Counter counter = testGroup.counter("test-counter");
339
counter.inc(); // Works locally but not reported
340

341
// Useful for unit tests
342
public class MyOperatorTest {
343
    @Test
344
    public void testOperator() {
345
        MetricGroup metrics = new UnregisteredMetricsGroup();
346
        MyOperator operator = new MyOperator(metrics);
347
        // Test operator logic without metric reporting overhead
348
    }
349
}
350
```