tessl/maven-org-hdrhistogram--hdr-histogram

High Dynamic Range (HDR) Histogram for recording and analyzing value distributions with configurable precision across wide dynamic ranges.

—

Pending

Overview

Eval results

Files

Double Value Histograms

Name: tessl/maven-org-hdrhistogram--hdr-histogram
Author: tessl

Histogram implementations for recording and analyzing double (floating-point) values with configurable precision and dynamic range. These histograms provide the same statistical analysis capabilities as integer histograms but for continuous-valued data.

DoubleHistogram

Core implementation for recording double values with HDR precision.

public class DoubleHistogram extends EncodableHistogram 
    implements DoubleValueRecorder, Serializable {
    
    // Constructors
    public DoubleHistogram(int numberOfSignificantValueDigits);
    public DoubleHistogram(long highestToLowestValueRatio, int numberOfSignificantValueDigits);
    public DoubleHistogram(int numberOfSignificantValueDigits, 
                          Class<? extends AbstractHistogram> internalCountsHistogramClass);
    
    // Factory methods
    static DoubleHistogram decodeFromByteBuffer(ByteBuffer buffer, 
                                               long minBarForHighestToLowestValueRatio);
    static DoubleHistogram decodeFromCompressedByteBuffer(ByteBuffer buffer, 
                                                         long minBarForHighestToLowestValueRatio);
    static DoubleHistogram fromString(String base64CompressedHistogramString);
    
    // Implementation methods
    public DoubleHistogram copy();
    public DoubleHistogram copyCorrectedForCoordinatedOmission(double expectedInterval);
}

Recording Double Values

// Core recording methods
void recordValue(double value);
void recordValueWithCount(double value, long count);
void recordValueWithExpectedInterval(double value, double expectedInterval);
void reset();

Usage Examples:

// Create double histogram with auto-resizing
DoubleHistogram histogram = new DoubleHistogram(3);

// Record floating-point measurements
histogram.recordValue(1.234);     // Response time in seconds
histogram.recordValue(0.987);     // Another measurement
histogram.recordValue(2.345);     // Higher latency

// Record multiple occurrences
histogram.recordValueWithCount(1.5, 10);  // Value 1.5 occurred 10 times

// Coordinated omission correction for timing measurements
double expectedInterval = 0.1;  // Expected 100ms intervals
double actualMeasurement = 1.5;  // But measured 1.5 seconds (pause occurred)
histogram.recordValueWithExpectedInterval(actualMeasurement, expectedInterval);

Statistical Analysis

// Count and statistics
long getTotalCount();
double getMaxValue();
double getMinValue();
double getMinNonZeroValue();
double getMean();
double getStdDeviation();

// Percentile queries
double getValueAtPercentile(double percentile);
double getPercentileAtOrBelowValue(double value);

// Value-specific queries
long getCountAtValue(double value);
long getCountBetweenValues(double lowValue, double highValue);

Usage Examples:

// Analyze response time distribution
long totalRequests = histogram.getTotalCount();
double avgResponseTime = histogram.getMean();
double maxResponseTime = histogram.getMaxValue();

// Key percentiles for SLA monitoring
double p50 = histogram.getValueAtPercentile(50.0);   // Median response time
double p95 = histogram.getValueAtPercentile(95.0);   // 95th percentile
double p99 = histogram.getValueAtPercentile(99.0);   // 99th percentile
double p999 = histogram.getValueAtPercentile(99.9);  // 99.9th percentile

System.out.printf("Response Time Analysis:%n");
System.out.printf("  Requests: %d%n", totalRequests);
System.out.printf("  Average: %.3f sec%n", avgResponseTime);
System.out.printf("  P50: %.3f sec%n", p50);
System.out.printf("  P95: %.3f sec%n", p95);
System.out.printf("  P99: %.3f sec%n", p99);
System.out.printf("  P999: %.3f sec%n", p999);

// SLA analysis - what percentage meets 2-second target?
double percentileAt2Sec = histogram.getPercentileAtOrBelowValue(2.0);
System.out.printf("%.2f%% of requests complete within 2 seconds%n", percentileAt2Sec);

Value Equivalence Analysis

// Value equivalence methods
double sizeOfEquivalentValueRange(double value);
double lowestEquivalentValue(double value);
double highestEquivalentValue(double value);
double medianEquivalentValue(double value);
double nextNonEquivalentValue(double value);
boolean valuesAreEquivalent(double value1, double value2);

Usage Examples:

// Understand histogram precision for double values
double value = 1.234;
double rangeSize = histogram.sizeOfEquivalentValueRange(value);
double lowest = histogram.lowestEquivalentValue(value);
double highest = histogram.highestEquivalentValue(value);
double median = histogram.medianEquivalentValue(value);

System.out.printf("Value %.6f represents range [%.6f, %.6f]%n", value, lowest, highest);
System.out.printf("Range size: %.6f%n", rangeSize);
System.out.printf("Median equivalent: %.6f%n", median);

// Check if two values would be considered equivalent
double val1 = 1.2345;
double val2 = 1.2346;
if (histogram.valuesAreEquivalent(val1, val2)) {
    System.out.println("Values are equivalent within histogram precision");
}

// Iterate through distinct values efficiently
double currentValue = histogram.getMinNonZeroValue();
while (currentValue <= histogram.getMaxValue()) {
    long count = histogram.getCountAtValue(currentValue);
    if (count > 0) {
        System.out.printf("Value: %.6f, Count: %d%n", currentValue, count);
    }
    currentValue = histogram.nextNonEquivalentValue(currentValue);
}

Configuration and Auto-Resize

// Configuration
void setAutoResize(boolean autoResize);
boolean isAutoResize();
long getHighestToLowestValueRatio();
int getNumberOfSignificantValueDigits();

// Timestamp and metadata
long getStartTimeStamp();
void setStartTimeStamp(long timestamp);
long getEndTimeStamp();
void setEndTimeStamp(long timestamp);
String getTag();
void setTag(String tag);

Usage Examples:

// Create histogram with specific dynamic range
long dynamicRange = 1000000;  // 1M:1 ratio (6 orders of magnitude)
DoubleHistogram histogram = new DoubleHistogram(dynamicRange, 3);

// Enable auto-resize for unknown ranges
histogram.setAutoResize(true);

// Tag histogram for identification
histogram.setTag("api-response-times");
histogram.setStartTimeStamp(System.currentTimeMillis());

// Record measurements
recordApiResponseTimes(histogram);

histogram.setEndTimeStamp(System.currentTimeMillis());
System.out.printf("Measurement period: %s%n", histogram.getTag());

Constructor Variations

// Auto-resizing histogram (recommended for most use cases)
DoubleHistogram autoResize = new DoubleHistogram(3);

// Fixed dynamic range (1M:1 ratio)
DoubleHistogram fixedRange = new DoubleHistogram(1_000_000, 3);

// Custom internal histogram implementation
DoubleHistogram withIntCounts = new DoubleHistogram(3, IntCountsHistogram.class);
DoubleHistogram withPackedStorage = new DoubleHistogram(3, PackedHistogram.class);

ConcurrentDoubleHistogram

Thread-safe version of DoubleHistogram supporting concurrent recording operations.

public class ConcurrentDoubleHistogram extends DoubleHistogram {
    
    // Constructors
    public ConcurrentDoubleHistogram(int numberOfSignificantValueDigits);
    public ConcurrentDoubleHistogram(long highestToLowestValueRatio, int numberOfSignificantValueDigits);
    public ConcurrentDoubleHistogram(int numberOfSignificantValueDigits,
                                    Class<? extends AbstractHistogram> internalCountsHistogramClass);
    
    // Implementation methods
    public ConcurrentDoubleHistogram copy();
    public ConcurrentDoubleHistogram copyCorrectedForCoordinatedOmission(double expectedInterval);
}

Thread Safety Characteristics

Wait-Free Operations:

recordValue() - Concurrent recording without contention
recordValueWithCount() - Concurrent recording with counts
recordValueWithExpectedInterval() - Concurrent coordinated omission correction
Auto-resizing operations

Requires External Synchronization:

Statistical queries and analysis
Iterator operations
Histogram manipulation operations

Usage Examples

// Create concurrent double histogram for multi-threaded recording
ConcurrentDoubleHistogram histogram = new ConcurrentDoubleHistogram(3);

// Multiple threads recording response times concurrently
ExecutorService requestProcessors = Executors.newFixedThreadPool(10);

for (int i = 0; i < 10; i++) {
    requestProcessors.submit(() -> {
        Random random = new Random();
        for (int j = 0; j < 10000; j++) {
            // Simulate processing and record response time
            double startTime = System.nanoTime() / 1e9;
            simulateApiCall();  // Your API processing
            double endTime = System.nanoTime() / 1e9;
            
            // Thread-safe recording (no synchronization needed)
            histogram.recordValue(endTime - startTime);
        }
    });
}

// Separate monitoring thread
ScheduledExecutorService monitor = Executors.newSingleThreadScheduledExecutor();
monitor.scheduleAtFixedRate(() -> {
    // Synchronized reading for consistent snapshot
    synchronized (histogram) {
        double p95 = histogram.getValueAtPercentile(95.0);
        double p99 = histogram.getValueAtPercentile(99.0);
        System.out.printf("Current P95: %.3fs, P99: %.3fs%n", p95, p99);
    }
}, 0, 10, TimeUnit.SECONDS);

requestProcessors.shutdown();
monitor.shutdown();

Writer-Reader Coordination

For more sophisticated coordination without blocking:

ConcurrentDoubleHistogram histogram = new ConcurrentDoubleHistogram(3);
WriterReaderPhaser phaser = histogram.getWriterReaderPhaser();

// Recording threads (multiple)
Runnable recorder = () -> {
    Random random = new Random();
    for (int i = 0; i < 100000; i++) {
        double measurement = random.nextGaussian() * 0.1 + 0.5;  // ~500ms ± 100ms
        histogram.recordValue(measurement);
    }
};

// Analysis thread (single)
Runnable analyzer = () -> {
    phaser.readerLock();
    try {
        // Safe to iterate and analyze
        System.out.printf("Total count: %d%n", histogram.getTotalCount());
        histogram.outputPercentileDistribution(System.out, 1.0);
    } finally {
        phaser.readerUnlock();
    }
};

// Start concurrent operations
new Thread(recorder).start();
new Thread(recorder).start();
new Thread(analyzer).start();

SynchronizedDoubleHistogram

Fully thread-safe double histogram with synchronized access for all operations.

public class SynchronizedDoubleHistogram extends DoubleHistogram {
    
    // Constructors  
    public SynchronizedDoubleHistogram(int numberOfSignificantValueDigits);
    public SynchronizedDoubleHistogram(long highestToLowestValueRatio, int numberOfSignificantValueDigits);
    public SynchronizedDoubleHistogram(int numberOfSignificantValueDigits,
                                      Class<? extends AbstractHistogram> internalCountsHistogramClass);
    
    // Implementation methods
    public SynchronizedDoubleHistogram copy();
    public SynchronizedDoubleHistogram copyCorrectedForCoordinatedOmission(double expectedInterval);
}

Usage Examples

// Fully synchronized double histogram
SynchronizedDoubleHistogram histogram = new SynchronizedDoubleHistogram(3);

// All operations are thread-safe but may block
Runnable measurements = () -> {
    Random random = new Random();
    for (int i = 0; i < 50000; i++) {
        double value = Math.abs(random.nextGaussian()) * 2.0;  // Positive values 0-6 range
        histogram.recordValue(value);  // Thread-safe, may block
    }
};

Runnable monitoring = () -> {
    while (true) {
        // No synchronization needed - all operations are atomic
        long count = histogram.getTotalCount();
        double mean = histogram.getMean();
        double p95 = histogram.getValueAtPercentile(95.0);
        
        System.out.printf("Count: %d, Mean: %.3f, P95: %.3f%n", count, mean, p95);
        
        try { Thread.sleep(2000); } catch (InterruptedException e) { break; }
    }
};

// Start concurrent operations  
new Thread(measurements).start();
new Thread(measurements).start();
new Thread(monitoring).start();

DoubleValueRecorder Interface

Core interface for double value recording functionality.

public interface DoubleValueRecorder {
    void recordValue(double value);
    void recordValueWithCount(double value, long count);
    void recordValueWithExpectedInterval(double value, double expectedInterval);
    void reset();
}

Double Histogram Iteration

DoubleHistogram provides specialized iterators for double values:

// Double-specific iterators
DoublePercentileIterator percentileIterator(int percentileTicksPerHalfDistance);
DoubleLinearIterator linearIterator(double valueUnitsPerBucket);
DoubleLogarithmicIterator logarithmicIterator(double valueUnitsInFirstBucket, double logBase);
DoubleRecordedValuesIterator recordedValuesIterator();
DoubleAllValuesIterator allValuesIterator();

Usage Examples:

// Iterate through percentiles
DoublePercentileIterator percentileIter = histogram.percentileIterator(5);
while (percentileIter.hasNext()) {
    DoubleHistogramIterationValue value = percentileIter.next();
    System.out.printf("Percentile %.2f: %.6f (count: %d)%n",
        value.getPercentileLevelIteratedTo(),
        value.getValueIteratedTo(),
        value.getCountAtValueIteratedTo());
}

// Linear iteration through value ranges
DoubleLinearIterator linearIter = histogram.linearIterator(0.1);  // 100ms buckets
while (linearIter.hasNext()) {
    DoubleHistogramIterationValue value = linearIter.next();
    if (value.getCountAtValueIteratedTo() > 0) {
        System.out.printf("Range %.3f-%.3f: %d samples%n",
            value.getValueIteratedFrom(),
            value.getValueIteratedTo(),
            value.getTotalCountToThisValue());
    }
}

Practical Use Cases

Web Application Response Times

// Track API endpoint response times
ConcurrentDoubleHistogram apiResponseTimes = new ConcurrentDoubleHistogram(3);
apiResponseTimes.setTag("api-latency");

// In your request handler
@GetMapping("/api/data")
public ResponseEntity<Data> getData() {
    double startTime = System.nanoTime() / 1e9;
    
    try {
        Data result = dataService.getData();
        return ResponseEntity.ok(result);
    } finally {
        double endTime = System.nanoTime() / 1e9;
        apiResponseTimes.recordValue(endTime - startTime);
    }
}

// Periodic reporting
@Scheduled(fixedRate = 60000)  // Every minute
public void reportMetrics() {
    synchronized (apiResponseTimes) {
        double p95 = apiResponseTimes.getValueAtPercentile(95.0);
        double p99 = apiResponseTimes.getValueAtPercentile(99.0);
        
        if (p95 > 2.0) {  // Alert if P95 > 2 seconds
            alertingService.alert("API P95 latency high: " + p95 + "s");
        }
        
        metricsService.record("api.p95", p95);
        metricsService.record("api.p99", p99);
    }
}

Financial Transaction Processing

// Track transaction processing times with high precision
DoubleHistogram transactionTimes = new DoubleHistogram(4);  // High precision
transactionTimes.setTag("transaction-processing");

// Record transaction with coordinated omission correction
public void processTransaction(Transaction tx) {
    double expectedProcessingTime = 0.050;  // Expected 50ms processing
    double startTime = System.nanoTime() / 1e9;
    
    try {
        // Process transaction
        processTransactionInternal(tx);
    } finally {
        double endTime = System.nanoTime() / 1e9;
        double actualTime = endTime - startTime;
        
        // Account for coordinated omission (system pauses, GC, etc.)
        transactionTimes.recordValueWithExpectedInterval(actualTime, expectedProcessingTime);
    }
}

// Compliance reporting
public ComplianceReport generateComplianceReport() {
    return ComplianceReport.builder()
        .totalTransactions(transactionTimes.getTotalCount())
        .averageProcessingTime(transactionTimes.getMean())
        .p95ProcessingTime(transactionTimes.getValueAtPercentile(95.0))
        .p99ProcessingTime(transactionTimes.getValueAtPercentile(99.0))
        .maxProcessingTime(transactionTimes.getMaxValue())
        .complianceThresholdMet(transactionTimes.getValueAtPercentile(99.0) < 0.100)  // < 100ms P99
        .build();
}

Performance Considerations

Memory Usage

DoubleHistogram internally uses an integer histogram scaled to the appropriate range:

Memory usage similar to equivalent integer histogram
Dynamic range affects memory requirements
Consider using packed variants for sparse double distributions

Precision and Range

The numberOfSignificantValueDigits parameter affects both precision and memory:

2 digits: ~10% precision, low memory
3 digits: ~1% precision, moderate memory (recommended)
4 digits: ~0.1% precision, higher memory
5 digits: ~0.01% precision, high memory

Double Value Scaling

Double values are internally scaled to integers:

Very small values (< 1e-12) may lose precision
Very large values (> 1e12) may lose precision
Choose appropriate dynamic range for your data

PackedDoubleHistogram

Memory-optimized version of DoubleHistogram using packed array representation for sparse double value distributions.

public class PackedDoubleHistogram extends DoubleHistogram {
    
    // Constructors
    public PackedDoubleHistogram(int numberOfSignificantValueDigits);
    public PackedDoubleHistogram(long highestToLowestValueRatio, int numberOfSignificantValueDigits);
    public PackedDoubleHistogram(DoubleHistogram source);
    
    // Factory methods
    static PackedDoubleHistogram decodeFromByteBuffer(ByteBuffer buffer, 
                                                     long minBarForHighestToLowestValueRatio);
    static PackedDoubleHistogram decodeFromCompressedByteBuffer(ByteBuffer buffer, 
                                                               long minBarForHighestToLowestValueRatio);
}

Usage Examples

// Create packed double histogram for sparse distributions
PackedDoubleHistogram histogram = new PackedDoubleHistogram(3);

// Record sparse response time data (most values 0.1-1.0s, few outliers)
Random random = new Random();
for (int i = 0; i < 100000; i++) {
    if (random.nextDouble() < 0.9) {
        // 90% of values in normal range
        histogram.recordValue(0.1 + random.nextDouble() * 0.9);
    } else {
        // 10% outliers
        histogram.recordValue(5.0 + random.nextDouble() * 95.0);  
    }
}

// Packed storage provides memory efficiency for sparse distributions
System.out.printf("Memory usage: %d bytes%n", histogram.getEstimatedFootprintInBytes());

PackedConcurrentDoubleHistogram

Thread-safe packed double histogram combining memory efficiency with concurrent recording support.

public class PackedConcurrentDoubleHistogram extends ConcurrentDoubleHistogram {
    
    // Constructors
    public PackedConcurrentDoubleHistogram(int numberOfSignificantValueDigits);
    public PackedConcurrentDoubleHistogram(long highestToLowestValueRatio, int numberOfSignificantValueDigits);
    public PackedConcurrentDoubleHistogram(DoubleHistogram source);
    
    // Factory methods
    static PackedConcurrentDoubleHistogram decodeFromByteBuffer(ByteBuffer buffer, 
                                                               long minBarForHighestToLowestValueRatio);
    static PackedConcurrentDoubleHistogram decodeFromCompressedByteBuffer(ByteBuffer buffer, 
                                                                         long minBarForHighestToLowestValueRatio);
    
    // Implementation methods
    public PackedConcurrentDoubleHistogram copy();
    public PackedConcurrentDoubleHistogram copyCorrectedForCoordinatedOmission(double expectedInterval);
}

Usage Examples

// Thread-safe packed double histogram for concurrent sparse recording
PackedConcurrentDoubleHistogram histogram = new PackedConcurrentDoubleHistogram(3);

// Multiple threads recording sparse latency measurements
ExecutorService executor = Executors.newFixedThreadPool(8);

for (int t = 0; t < 8; t++) {
    executor.submit(() -> {
        Random random = new Random();
        for (int i = 0; i < 50000; i++) {
            // Simulate API latency: mostly fast, occasional slow responses
            double latency = random.nextDouble() < 0.95
                ? 0.001 + random.nextDouble() * 0.099    // 95% fast: 1-100ms
                : 1.0 + random.nextDouble() * 9.0;       // 5% slow: 1-10s
                
            histogram.recordValue(latency);  // Thread-safe recording
        }
    });
}

// Coordinated analysis
WriterReaderPhaser phaser = histogram.getWriterReaderPhaser();
phaser.readerLock();
try {
    System.out.printf("Packed memory usage: %d bytes%n", 
        histogram.getEstimatedFootprintInBytes());
    
    double p95 = histogram.getValueAtPercentile(95.0);
    double p99 = histogram.getValueAtPercentile(99.0);
    
    System.out.printf("P95: %.3fs, P99: %.3fs%n", p95, p99);
} finally {
    phaser.readerUnlock();
}

executor.shutdown();

Thread Safety Summary

Histogram Type	Recording Thread Safety	Query Thread Safety	Auto-Resize Support
DoubleHistogram	No	No	Yes
ConcurrentDoubleHistogram	Yes (wait-free)	External sync needed	Yes
SynchronizedDoubleHistogram	Yes (synchronized)	Yes (synchronized)	Yes
PackedDoubleHistogram	No	No	Yes
PackedConcurrentDoubleHistogram	Yes (wait-free)	External sync needed	Yes

Install with Tessl CLI

npx tessl i tessl/maven-org-hdrhistogram--hdr-histogram

docs

concurrent-histograms.md

specialized-variants.md

utilities.md

tile.json

tessl/maven-org-hdrhistogram--hdr-histogram

double-histograms.md.css-3qkkll{font-size:var(--chakra-font-sizes-sm);font-weight:var(--chakra-font-weights-normal);color:var(--chakra-colors-gray-300);}docs/

Double Value Histograms

DoubleHistogram

Recording Double Values

Statistical Analysis

Value Equivalence Analysis

Configuration and Auto-Resize

Constructor Variations

ConcurrentDoubleHistogram

Thread Safety Characteristics

Usage Examples

Writer-Reader Coordination

SynchronizedDoubleHistogram

Usage Examples

DoubleValueRecorder Interface

Double Histogram Iteration

Practical Use Cases

Web Application Response Times

Financial Transaction Processing

Performance Considerations

Memory Usage

Precision and Range

Double Value Scaling

PackedDoubleHistogram

Usage Examples

PackedConcurrentDoubleHistogram

Usage Examples

Thread Safety Summary

double-histograms.mddocs/