0
# Utility Classes
1
2
Supporting classes providing specialized functionality for synchronization, encoding, data transformation, and low-level operations within the HdrHistogram ecosystem.
3
4
## WriterReaderPhaser
5
6
Asymmetric synchronization primitive enabling wait-free writers with phase-flipping readers. This utility coordinates concurrent recording with non-blocking reader phases.
7
8
```java { .api }
9
public class WriterReaderPhaser {
10
11
// Constructor
12
public WriterReaderPhaser();
13
14
// Writer methods (wait-free)
15
long writerCriticalSectionEnter();
16
void writerCriticalSectionExit(long criticalValueAtEnter);
17
18
// Reader methods (blocking for other readers)
19
void readerLock();
20
void readerUnlock();
21
22
// Phase management
23
void flipPhase();
24
void flipPhase(long sleepTimeBeforeFlip);
25
26
// State queries
27
long getEpoch();
28
}
29
```
30
31
### Writer-Reader Coordination
32
33
The WriterReaderPhaser enables a sophisticated coordination pattern:
34
35
- **Writers**: Enter and exit critical sections without blocking (wait-free)
36
- **Readers**: Acquire exclusive access for consistent data snapshots
37
- **Phase Flipping**: Coordinates between active and inactive data structures
38
39
```java
40
// Example: Coordinated histogram access pattern
41
public class CoordinatedHistogramAccess {
42
43
private final WriterReaderPhaser phaser = new WriterReaderPhaser();
44
private volatile Histogram activeHistogram;
45
private volatile Histogram inactiveHistogram;
46
47
// Wait-free recording (multiple writers supported)
48
public void recordValue(long value) {
49
long criticalValue = phaser.writerCriticalSectionEnter();
50
try {
51
// Record to current active histogram
52
activeHistogram.recordValue(value);
53
} finally {
54
phaser.writerCriticalSectionExit(criticalValue);
55
}
56
}
57
58
// Coordinated reading with phase flip
59
public Histogram getIntervalHistogram() {
60
phaser.readerLock();
61
try {
62
// Swap active/inactive histograms
63
Histogram intervalHist = inactiveHistogram;
64
inactiveHistogram = activeHistogram;
65
activeHistogram = intervalHist;
66
67
// Flip phase to notify writers of the change
68
phaser.flipPhase();
69
70
// Clear the new inactive histogram for next interval
71
inactiveHistogram.reset();
72
73
return intervalHist; // Contains previous interval's data
74
} finally {
75
phaser.readerUnlock();
76
}
77
}
78
}
79
```
80
81
### Advanced Phaser Usage
82
83
```java
84
public class AdvancedPhaserUsage {
85
86
private final WriterReaderPhaser phaser = new WriterReaderPhaser();
87
private final AtomicLong totalRecordings = new AtomicLong();
88
89
public void demonstratePhaserPatterns() {
90
// Multiple concurrent writers
91
ExecutorService writers = Executors.newFixedThreadPool(4);
92
93
for (int i = 0; i < 4; i++) {
94
writers.submit(() -> {
95
Random random = new Random();
96
97
for (int j = 0; j < 100000; j++) {
98
// Wait-free writer pattern
99
long criticalValue = phaser.writerCriticalSectionEnter();
100
try {
101
// Simulate work with shared data structure
102
performWriterWork(random.nextInt(1000));
103
totalRecordings.incrementAndGet();
104
} finally {
105
phaser.writerCriticalSectionExit(criticalValue);
106
}
107
}
108
});
109
}
110
111
// Single reader with periodic phase flips
112
ScheduledExecutorService reader = Executors.newSingleThreadScheduledExecutor();
113
114
reader.scheduleAtFixedRate(() -> {
115
phaser.readerLock();
116
try {
117
long currentEpoch = phaser.getEpoch();
118
long recordings = totalRecordings.get();
119
120
System.out.printf("Epoch %d: %d total recordings%n", currentEpoch, recordings);
121
122
// Flip phase for next measurement interval
123
phaser.flipPhase(10); // Brief pause before flip
124
125
} finally {
126
phaser.readerUnlock();
127
}
128
}, 1, 1, TimeUnit.SECONDS);
129
130
// Cleanup
131
writers.shutdown();
132
reader.shutdown();
133
}
134
}
135
```
136
137
## Base64Helper
138
139
Utility class for Base64 encoding and decoding of histogram data, enabling text-based storage and transmission.
140
141
```java { .api }
142
public class Base64Helper {
143
144
// Encoding methods
145
static String printBase64Binary(byte[] bytes);
146
static String printBase64Binary(byte[] bytes, int offset, int length);
147
148
// Decoding methods
149
static byte[] parseBase64Binary(String base64String);
150
}
151
```
152
153
### Basic Base64 Operations
154
155
```java
156
// Encode binary data to Base64 string
157
byte[] histogramData = getHistogramBinaryData();
158
String base64Encoded = Base64Helper.printBase64Binary(histogramData);
159
160
System.out.println("Base64 encoded histogram:");
161
System.out.println(base64Encoded);
162
163
// Decode Base64 string back to binary
164
byte[] decodedData = Base64Helper.parseBase64Binary(base64Encoded);
165
166
// Verify integrity
167
boolean isIdentical = Arrays.equals(histogramData, decodedData);
168
System.out.printf("Data integrity preserved: %s%n", isIdentical);
169
```
170
171
### Histogram Base64 Serialization
172
173
```java
174
public class HistogramBase64Serialization {
175
176
public String serializeHistogram(AbstractHistogram histogram) {
177
try {
178
// Get compressed binary representation
179
int bufferSize = histogram.getNeededByteBufferCapacity();
180
ByteBuffer buffer = ByteBuffer.allocate(bufferSize);
181
182
int compressedSize = histogram.encodeIntoCompressedByteBuffer(buffer, 6);
183
184
// Extract compressed bytes
185
buffer.flip();
186
byte[] compressedData = new byte[compressedSize];
187
buffer.get(compressedData);
188
189
// Encode to Base64
190
return Base64Helper.printBase64Binary(compressedData);
191
192
} catch (Exception e) {
193
throw new RuntimeException("Failed to serialize histogram", e);
194
}
195
}
196
197
public Histogram deserializeHistogram(String base64Data) {
198
try {
199
// Decode from Base64
200
byte[] compressedData = Base64Helper.parseBase64Binary(base64Data);
201
202
// Decompress and decode histogram
203
ByteBuffer buffer = ByteBuffer.wrap(compressedData);
204
return Histogram.decodeFromCompressedByteBuffer(buffer, 1000);
205
206
} catch (Exception e) {
207
throw new RuntimeException("Failed to deserialize histogram", e);
208
}
209
}
210
211
// Example usage in configuration or messaging
212
public void demonstrateBase64Usage() {
213
// Create sample histogram
214
Histogram original = new Histogram(3);
215
Random random = new Random();
216
217
for (int i = 0; i < 10000; i++) {
218
original.recordValue(random.nextInt(10000));
219
}
220
221
// Serialize to Base64
222
String serialized = serializeHistogram(original);
223
System.out.printf("Serialized histogram length: %d characters%n", serialized.length());
224
225
// Deserialize from Base64
226
Histogram restored = deserializeHistogram(serialized);
227
228
// Verify equivalence
229
if (original.equals(restored)) {
230
System.out.println("✓ Base64 serialization preserves histogram data");
231
}
232
233
// Show usage in JSON-like structures
234
System.out.printf("JSON representation: {\"histogram\": \"%s\"}%n",
235
serialized.substring(0, Math.min(50, serialized.length())) + "...");
236
}
237
}
238
```
239
240
## ZigZagEncoding
241
242
Utility class for ZigZag encoding and decoding, used internally for efficient variable-length integer serialization.
243
244
```java { .api }
245
public class ZigZagEncoding {
246
247
// Long encoding/decoding
248
static void putLong(ByteBuffer buffer, long value);
249
static long getLong(ByteBuffer buffer);
250
251
// Integer encoding/decoding
252
static void putInt(ByteBuffer buffer, int value);
253
static int getInt(ByteBuffer buffer);
254
}
255
```
256
257
### ZigZag Encoding Principles
258
259
ZigZag encoding maps signed integers to unsigned integers efficiently:
260
- Small positive and negative numbers use fewer bytes
261
- Alternates positive and negative values: 0, -1, 1, -2, 2, -3, 3, ...
262
- Enables efficient variable-length encoding
263
264
```java
265
public class ZigZagEncodingDemo {
266
267
public void demonstrateZigZagEncoding() {
268
// Test various values to show encoding efficiency
269
long[] testValues = {0, -1, 1, -100, 100, -10000, 10000, Long.MAX_VALUE, Long.MIN_VALUE};
270
271
ByteBuffer buffer = ByteBuffer.allocate(1024);
272
273
System.out.println("ZigZag Encoding Demonstration:");
274
System.out.println("Original -> Encoded -> Decoded");
275
276
for (long value : testValues) {
277
buffer.clear();
278
279
// Encode value
280
ZigZagEncoding.putLong(buffer, value);
281
int encodedBytes = buffer.position();
282
283
// Decode value
284
buffer.flip();
285
long decoded = ZigZagEncoding.getLong(buffer);
286
287
System.out.printf("%12d -> %d bytes -> %12d %s%n",
288
value, encodedBytes, decoded,
289
value == decoded ? "✓" : "✗");
290
}
291
}
292
293
public void compareEncodingEfficiency() {
294
Random random = new Random();
295
ByteBuffer zigzagBuffer = ByteBuffer.allocate(10000);
296
ByteBuffer standardBuffer = ByteBuffer.allocate(10000);
297
298
// Test with small values (common in histogram contexts)
299
for (int i = 0; i < 1000; i++) {
300
long value = random.nextInt(10000) - 5000; // Range: -5000 to +5000
301
302
// ZigZag encoding
303
zigzagBuffer.clear();
304
ZigZagEncoding.putLong(zigzagBuffer, value);
305
int zigzagBytes = zigzagBuffer.position();
306
307
// Standard encoding
308
standardBuffer.clear();
309
standardBuffer.putLong(value);
310
int standardBytes = standardBuffer.position();
311
312
if (i < 10) { // Show first 10 examples
313
System.out.printf("Value %6d: ZigZag=%d bytes, Standard=%d bytes%n",
314
value, zigzagBytes, standardBytes);
315
}
316
}
317
}
318
}
319
```
320
321
### Custom ZigZag Applications
322
323
```java
324
public class CustomZigZagApplications {
325
326
// Efficient delta encoding using ZigZag
327
public byte[] encodeDeltaSequence(long[] values) {
328
ByteBuffer buffer = ByteBuffer.allocate(values.length * 10); // Generous allocation
329
330
long previousValue = 0;
331
for (long value : values) {
332
long delta = value - previousValue;
333
ZigZagEncoding.putLong(buffer, delta);
334
previousValue = value;
335
}
336
337
// Return compact byte array
338
byte[] result = new byte[buffer.position()];
339
buffer.flip();
340
buffer.get(result);
341
return result;
342
}
343
344
public long[] decodeDeltaSequence(byte[] encoded) {
345
ByteBuffer buffer = ByteBuffer.wrap(encoded);
346
List<Long> values = new ArrayList<>();
347
348
long currentValue = 0;
349
while (buffer.hasRemaining()) {
350
long delta = ZigZagEncoding.getLong(buffer);
351
currentValue += delta;
352
values.add(currentValue);
353
}
354
355
return values.stream().mapToLong(Long::longValue).toArray();
356
}
357
358
// Demonstrate delta encoding efficiency
359
public void demonstrateDeltaEncoding() {
360
// Create sequence with small increments (typical in histogram buckets)
361
long[] sequence = new long[100];
362
long current = 1000;
363
Random random = new Random();
364
365
for (int i = 0; i < sequence.length; i++) {
366
current += random.nextInt(10) + 1; // Small increments
367
sequence[i] = current;
368
}
369
370
// Standard encoding
371
int standardSize = sequence.length * 8; // 8 bytes per long
372
373
// Delta ZigZag encoding
374
byte[] deltaEncoded = encodeDeltaSequence(sequence);
375
376
System.out.printf("Sequence encoding comparison:%n");
377
System.out.printf(" Standard: %d bytes%n", standardSize);
378
System.out.printf(" Delta ZigZag: %d bytes%n", deltaEncoded.length);
379
System.out.printf(" Compression ratio: %.2f%n",
380
(double) deltaEncoded.length / standardSize);
381
382
// Verify correctness
383
long[] decoded = decodeDeltaSequence(deltaEncoded);
384
boolean correct = Arrays.equals(sequence, decoded);
385
System.out.printf(" Decoding correctness: %s%n", correct ? "✓" : "✗");
386
}
387
}
388
```
389
390
## PackedArray Utilities
391
392
Supporting classes for memory-efficient sparse array storage used by PackedHistogram variants.
393
394
### AbstractPackedArrayContext
395
396
```java { .api }
397
public abstract class AbstractPackedArrayContext {
398
399
// Array management
400
abstract void setVirtualLength(int virtualLength);
401
abstract int getVirtualLength();
402
403
// Value access
404
abstract long getValueAtIndex(int index);
405
abstract void setValueAtIndex(int index, long value);
406
abstract long incrementValue(int index, long increment);
407
408
// Bulk operations
409
abstract int getPopulatedShortLength();
410
abstract boolean hasIndex(int index);
411
abstract void clear();
412
413
// Memory management
414
abstract void resize(int newVirtualLength);
415
abstract long getPhysicalLength();
416
}
417
```
418
419
### PackedArrayContext
420
421
Non-concurrent implementation for single-threaded packed array operations.
422
423
```java { .api }
424
public class PackedArrayContext extends AbstractPackedArrayContext {
425
426
// Constructors
427
public PackedArrayContext(int virtualLength);
428
public PackedArrayContext(int virtualLength, int initialPhysicalLength);
429
430
// Copy operations
431
void copyAndIncrement(AbstractPackedArrayContext sourceContext,
432
AbstractPackedArrayContext targetContext);
433
}
434
```
435
436
### ConcurrentPackedArrayContext
437
438
Thread-safe implementation supporting concurrent packed array operations.
439
440
```java { .api }
441
public class ConcurrentPackedArrayContext extends AbstractPackedArrayContext {
442
443
// Constructors
444
public ConcurrentPackedArrayContext(int virtualLength);
445
public ConcurrentPackedArrayContext(int virtualLength, int initialPhysicalLength);
446
447
// Thread-safe operations
448
@Override
449
public long incrementValue(int index, long increment);
450
451
// Concurrent copy operations
452
void copyAndIncrement(AbstractPackedArrayContext sourceContext,
453
AbstractPackedArrayContext targetContext);
454
}
455
```
456
457
### Packed Array Usage Examples
458
459
```java
460
public class PackedArrayDemo {
461
462
public void demonstratePackedArrayEfficiency() {
463
int virtualLength = 1000000; // 1M possible indices
464
465
// Create packed array contexts
466
PackedArrayContext packedContext = new PackedArrayContext(virtualLength);
467
long[] standardArray = new long[virtualLength]; // For comparison
468
469
// Populate with sparse data (only 1% of indices used)
470
Random random = new Random();
471
Set<Integer> usedIndices = new HashSet<>();
472
473
for (int i = 0; i < virtualLength / 100; i++) { // 1% population
474
int index = random.nextInt(virtualLength);
475
long value = random.nextInt(1000) + 1;
476
477
packedContext.setValueAtIndex(index, value);
478
standardArray[index] = value;
479
usedIndices.add(index);
480
}
481
482
// Compare memory usage
483
long standardMemory = virtualLength * 8L; // 8 bytes per long
484
long packedMemory = packedContext.getPhysicalLength() * 8L; // Actual storage
485
486
System.out.printf("Memory usage comparison:%n");
487
System.out.printf(" Standard array: %,d bytes%n", standardMemory);
488
System.out.printf(" Packed array: %,d bytes%n", packedMemory);
489
System.out.printf(" Memory savings: %.1f%% (%,d bytes saved)%n",
490
100.0 * (standardMemory - packedMemory) / standardMemory,
491
standardMemory - packedMemory);
492
System.out.printf(" Populated indices: %,d / %,d (%.2f%%)%n",
493
usedIndices.size(), virtualLength, 100.0 * usedIndices.size() / virtualLength);
494
495
// Verify data integrity
496
boolean dataIntegrity = true;
497
for (int index : usedIndices) {
498
if (packedContext.getValueAtIndex(index) != standardArray[index]) {
499
dataIntegrity = false;
500
break;
501
}
502
}
503
504
System.out.printf(" Data integrity: %s%n", dataIntegrity ? "✓" : "✗");
505
}
506
507
public void demonstrateConcurrentPackedArray() {
508
int virtualLength = 100000;
509
ConcurrentPackedArrayContext context = new ConcurrentPackedArrayContext(virtualLength);
510
511
// Multiple threads incrementing sparse indices
512
ExecutorService executor = Executors.newFixedThreadPool(4);
513
CountDownLatch latch = new CountDownLatch(4);
514
515
for (int t = 0; t < 4; t++) {
516
final int threadId = t;
517
executor.submit(() -> {
518
try {
519
Random random = new Random(threadId); // Different seed per thread
520
521
for (int i = 0; i < 10000; i++) {
522
int index = random.nextInt(1000); // Concentrate on first 1000 indices
523
context.incrementValue(index, 1); // Thread-safe increment
524
}
525
} finally {
526
latch.countDown();
527
}
528
});
529
}
530
531
try {
532
latch.await();
533
} catch (InterruptedException e) {
534
Thread.currentThread().interrupt();
535
}
536
537
// Analyze results
538
long totalIncrements = 0;
539
int populatedIndices = 0;
540
541
for (int i = 0; i < 1000; i++) {
542
long value = context.getValueAtIndex(i);
543
if (value > 0) {
544
totalIncrements += value;
545
populatedIndices++;
546
}
547
}
548
549
System.out.printf("Concurrent packed array results:%n");
550
System.out.printf(" Total increments: %,d%n", totalIncrements);
551
System.out.printf(" Populated indices: %d / 1000%n", populatedIndices);
552
System.out.printf(" Physical memory used: %,d bytes%n",
553
context.getPhysicalLength() * 8);
554
555
executor.shutdown();
556
}
557
}
558
```
559
560
## Performance Monitoring Utilities
561
562
### HistogramLogScanner
563
564
Utility for scanning histogram log files to extract metadata and time ranges.
565
566
```java { .api }
567
public class HistogramLogScanner {
568
569
// Constructor
570
public HistogramLogScanner(String fileName);
571
572
// Scanning methods
573
void scan();
574
double getStartTimeSec();
575
double getEndTimeSec();
576
long getTotalCount();
577
int getHistogramCount();
578
579
// Analysis methods
580
void outputTimeRange();
581
void outputSummary();
582
}
583
```
584
585
### Advanced Utility Patterns
586
587
```java
588
public class AdvancedUtilityPatterns {
589
590
// Coordinated multi-histogram recording
591
public static class CoordinatedMultiHistogram {
592
private final WriterReaderPhaser phaser = new WriterReaderPhaser();
593
private final Map<String, Histogram> activeHistograms = new ConcurrentHashMap<>();
594
private final Map<String, Histogram> inactiveHistograms = new ConcurrentHashMap<>();
595
596
public void recordValue(String metric, long value) {
597
long criticalValue = phaser.writerCriticalSectionEnter();
598
try {
599
activeHistograms.computeIfAbsent(metric, k -> new Histogram(3))
600
.recordValue(value);
601
} finally {
602
phaser.writerCriticalSectionExit(criticalValue);
603
}
604
}
605
606
public Map<String, Histogram> getIntervalHistograms() {
607
phaser.readerLock();
608
try {
609
// Swap all histograms atomically
610
Map<String, Histogram> results = new HashMap<>(inactiveHistograms);
611
612
Map<String, Histogram> temp = inactiveHistograms;
613
inactiveHistograms.putAll(activeHistograms);
614
activeHistograms.clear();
615
activeHistograms.putAll(temp);
616
617
// Reset inactive histograms
618
inactiveHistograms.values().forEach(Histogram::reset);
619
620
phaser.flipPhase();
621
return results;
622
} finally {
623
phaser.readerUnlock();
624
}
625
}
626
}
627
628
// Efficient histogram serialization manager
629
public static class HistogramSerializationManager {
630
private final ThreadLocal<ByteBuffer> bufferCache = ThreadLocal.withInitial(
631
() -> ByteBuffer.allocate(65536) // 64KB per thread
632
);
633
634
public String serializeToBase64(AbstractHistogram histogram) {
635
ByteBuffer buffer = bufferCache.get();
636
buffer.clear();
637
638
// Ensure buffer is large enough
639
int needed = histogram.getNeededByteBufferCapacity();
640
if (buffer.capacity() < needed) {
641
buffer = ByteBuffer.allocate(needed);
642
bufferCache.set(buffer);
643
}
644
645
// Compress and encode
646
int compressedSize = histogram.encodeIntoCompressedByteBuffer(buffer, 6);
647
648
// Convert to Base64
649
buffer.flip();
650
byte[] compressed = new byte[compressedSize];
651
buffer.get(compressed);
652
653
return Base64Helper.printBase64Binary(compressed);
654
}
655
}
656
}
657
```
658
659
## Utility Class Selection Guide
660
661
### WriterReaderPhaser Use Cases
662
- **Concurrent recorder implementations**
663
- **Multi-threaded histogram access coordination**
664
- **Phase-based data collection systems**
665
- **Wait-free writer scenarios**
666
667
### Base64Helper Use Cases
668
- **JSON/XML histogram embedding**
669
- **Text-based configuration storage**
670
- **HTTP API histogram transmission**
671
- **Database text field storage**
672
673
### ZigZagEncoding Use Cases
674
- **Custom serialization protocols**
675
- **Delta compression implementations**
676
- **Variable-length integer encoding**
677
- **Network protocol optimization**
678
679
### PackedArray Classes Use Cases
680
- **Sparse data histogram implementations**
681
- **Memory-constrained environments**
682
- **Custom histogram variants**
683
- **Specialized count storage patterns**
684
685
## Performance Characteristics
686
687
| Utility | Performance Profile | Memory Usage | Thread Safety |
688
|---------|-------------------|--------------|---------------|
689
| WriterReaderPhaser | Wait-free writers, blocking readers | Minimal | Multi-threading coordination |
690
| Base64Helper | CPU-intensive encoding/decoding | Temporary allocation | Thread-safe (stateless) |
691
| ZigZagEncoding | Fast variable-length encoding | Minimal | Thread-safe (stateless) |
692
| PackedArrayContext | Fast sparse access | Dynamic compression | Single-threaded |
693
| ConcurrentPackedArrayContext | Slower but thread-safe | Dynamic compression | Thread-safe |