0
# Async Logging
1

2
Asynchronous logging capabilities for high-throughput applications with configurable queue sizes and discard policies. Async logging improves application performance by moving log I/O operations to background threads.
3

4
## Capabilities
5

6
### AsyncAppenderFactory Interface
7

8
Base interface for creating asynchronous appender wrappers that can wrap any synchronous appender in an async wrapper.
9

10
```java { .api }
11
/**
12
 * Interface for creating async appenders
13
 */
14
public interface AsyncAppenderFactory<E> {
15
    /**
16
     * Build an async appender wrapper
17
     * @return configured AsyncAppenderBase instance
18
     */
19
    AsyncAppenderBase<E> build();
20
}
21
```
22

23
### AsyncLoggingEventAppenderFactory
24

25
Default implementation that creates AsyncAppender instances specifically for ILoggingEvent objects.
26

27
```java { .api }
28
/**
29
 * Creates AsyncAppender instances for ILoggingEvent
30
 */
31
public class AsyncLoggingEventAppenderFactory implements AsyncAppenderFactory<ILoggingEvent> {
32
    /**
33
     * Build an AsyncAppender for logging events
34
     * @return configured AsyncAppender instance
35
     */
36
    @Override
37
    public AsyncAppenderBase<ILoggingEvent> build();
38
}
39
```
40

41
## Async Configuration in AbstractAppenderFactory
42

43
All appender factories support async configuration through the AbstractAppenderFactory base class:
44

45
```java { .api }
46
public abstract class AbstractAppenderFactory<E> implements AppenderFactory<E> {
47
    protected int queueSize = 256;
48
    protected int discardingThreshold = -1;
49
    protected boolean includeCallerData = false;
50
    protected boolean neverBlock = false;
51
    
52
    /**
53
     * Set the async queue size
54
     * @param queueSize the size of the async queue (default: 256)
55
     */
56
    public void setQueueSize(int queueSize);
57
    
58
    /**
59
     * Get the async queue size
60
     * @return the current queue size
61
     */
62
    public int getQueueSize();
63
    
64
    /**
65
     * Set the async discarding threshold
66
     * @param discardingThreshold events below this level may be discarded when queue is full
67
     *                           (-1 disables discarding, default: -1)
68
     */
69
    public void setDiscardingThreshold(int discardingThreshold);
70
    
71
    /**
72
     * Get the async discarding threshold
73
     * @return the current discarding threshold
74
     */
75
    public int getDiscardingThreshold();
76
    
77
    /**
78
     * Include caller data in async logging
79
     * @param includeCallerData true to include caller information (impacts performance)
80
     */
81
    public void setIncludeCallerData(boolean includeCallerData);
82
    
83
    /**
84
     * Check if caller data is included
85
     * @return true if caller data is included
86
     */
87
    public boolean isIncludeCallerData();
88
    
89
    /**
90
     * Set non-blocking behavior for async appenders
91
     * @param neverBlock true to never block on full queue (may lose log events)
92
     */
93
    public void setNeverBlock(boolean neverBlock);
94
    
95
    /**
96
     * Check if async appender never blocks
97
     * @return true if configured to never block
98
     */
99
    public boolean isNeverBlock();
100
    
101
    /**
102
     * Wrap appender in async wrapper if needed
103
     * @param context the logger context
104
     * @param asyncAppenderFactory factory for async appenders
105
     * @param appender the appender to wrap
106
     * @return the wrapped appender (or original if async not configured)
107
     */
108
    protected Appender<E> wrapAsync(LoggerContext context, 
109
                                   AsyncAppenderFactory<E> asyncAppenderFactory, 
110
                                   Appender<E> appender);
111
}
112
```
113

114
## Async Configuration Examples
115

116
### Basic Async File Logging
117

118
```java
119
FileAppenderFactory<ILoggingEvent> fileAppender = new FileAppenderFactory<>();
120
fileAppender.setCurrentLogFilename("./logs/application.log");
121

122
// Configure async settings
123
fileAppender.setQueueSize(512);               // Larger queue for high throughput
124
fileAppender.setIncludeCallerData(false);     // Disable for better performance
125
fileAppender.setNeverBlock(false);            // Block when queue is full (prevents loss)
126
fileAppender.setDiscardingThreshold(-1);     // Never discard events
127

128
// Build with async wrapper
129
LoggerContext context = (LoggerContext) LoggerFactory.getILoggerFactory();
130
AsyncLoggingEventAppenderFactory asyncFactory = new AsyncLoggingEventAppenderFactory();
131
Appender<ILoggingEvent> appender = fileAppender.build(context, "MyApp", 
132
    new DropwizardLayoutFactory(), new ThresholdLevelFilterFactory(), asyncFactory);
133
```
134

135
### High-Throughput Async Configuration
136

137
```java
138
ConsoleAppenderFactory<ILoggingEvent> consoleAppender = new ConsoleAppenderFactory<>();
139

140
// High-throughput configuration
141
consoleAppender.setQueueSize(2048);           // Very large queue
142
consoleAppender.setDiscardingThreshold(20);   // Discard INFO and below when queue is 80% full
143
consoleAppender.setIncludeCallerData(false);  // Disable caller data for performance
144
consoleAppender.setNeverBlock(true);          // Never block application threads
145

146
// This configuration prioritizes application performance over log completeness
147
```
148

149
### Balanced Async Configuration
150

151
```java
152
FileAppenderFactory<ILoggingEvent> fileAppender = new FileAppenderFactory<>();
153
fileAppender.setCurrentLogFilename("./logs/application.log");
154

155
// Balanced configuration
156
fileAppender.setQueueSize(1024);              // Reasonable queue size
157
fileAppender.setDiscardingThreshold(0);       // Discard TRACE events when queue is full
158
fileAppender.setIncludeCallerData(true);      // Include caller data for debugging
159
fileAppender.setNeverBlock(false);            // Block to ensure important events are logged
160

161
// This configuration balances performance with log completeness
162
```
163

164
## Async Behavior and Performance Considerations
165

166
### Queue Management
167

168
The async appender uses a bounded queue to buffer log events:
169

170
- **Queue Size**: Determines how many events can be buffered
171
- **Queue Full Behavior**: Controlled by `neverBlock` setting
172
  - `neverBlock = false`: Application thread blocks until queue space is available
173
  - `neverBlock = true`: Events may be dropped when queue is full
174

175
### Discarding Threshold
176

177
When the queue reaches the discarding threshold percentage:
178

179
- Events below the threshold level are discarded
180
- Higher-level events are still queued
181
- Threshold is calculated as: `(queueSize * discardingThreshold) / 100`
182
- Use `-1` to disable discarding entirely
183

184
**Discarding Level Mapping:**
185
```java
186
// Default discarding behavior by level
187
if (discardingThreshold >= 0) {
188
    // Events at TRACE, DEBUG levels may be discarded
189
    // INFO, WARN, ERROR levels are preserved
190
}
191
```
192

193
### Caller Data Performance Impact
194

195
Including caller data has significant performance implications:
196

197
```java
198
// High performance (recommended for production)
199
appender.setIncludeCallerData(false);
200

201
// Lower performance but includes method/line information
202
appender.setIncludeCallerData(true);
203
```
204

205
When `includeCallerData = true`, each log event captures:
206
- Method name
207
- Line number  
208
- Class name
209
- File name
210

211
This information is expensive to obtain and should be disabled in high-throughput scenarios.
212

213
### Async Appender Shutdown
214

215
Async appenders must be properly shut down to flush remaining events:
216

217
```java
218
// Proper shutdown
219
LoggingFactory loggingFactory = // ... your logging factory
220
loggingFactory.stop(); // Flushes async queues and stops background threads
221

222
// Or manually stop async appenders
223
AsyncAppenderBase<ILoggingEvent> asyncAppender = // ... your async appender
224
asyncAppender.stop(); // Waits for queue to drain and stops worker thread
225
```
226

227
## Monitoring Async Appenders
228

229
### Queue Statistics
230

231
You can monitor async appender performance through JMX or programmatically:
232

233
```java
234
AsyncAppenderBase<ILoggingEvent> asyncAppender = // ... your async appender
235

236
// Check queue status
237
int queueSize = asyncAppender.getQueueSize();
238
int remainingCapacity = asyncAppender.getRemainingCapacity();
239
int numberOfElementsInQueue = queueSize - remainingCapacity;
240

241
// Monitor discard counts (if supported by implementation)
242
// Note: Dropwizard may expose these through metrics
243
```
244

245
### Metrics Integration
246

247
Dropwizard Logging integrates with Dropwizard Metrics to provide async appender monitoring:
248

249
```java
250
MetricRegistry metrics = new MetricRegistry();
251
loggingFactory.configure(metrics, "MyApplication");
252

253
// Metrics may include:
254
// - Queue depth
255
// - Discard counts  
256
// - Processing rates
257
// - Error counts
258
```
259

260
## Troubleshooting Async Logging
261

262
### Common Issues
263

264
**Log Events Not Appearing:**
265
- Check if queue is full and events are being discarded
266
- Verify proper shutdown to flush remaining events
267
- Ensure async appender is properly started
268

269
**Application Blocking:**
270
- Check `neverBlock` setting
271
- Monitor queue depth
272
- Consider increasing queue size
273

274
**Missing Caller Information:**
275
- Verify `includeCallerData` is set to `true`
276
- Note performance impact of enabling caller data
277

278
**Memory Usage:**
279
- Large queue sizes consume more memory
280
- Each queued event holds references until processed
281
- Consider queue size vs. memory constraints
282

283
### Debug Configuration
284

285
```java
286
// Enable debug logging for async appenders
287
Logger asyncLogger = LoggerFactory.getLogger(AsyncAppender.class);
288
asyncLogger.setLevel(Level.DEBUG);
289

290
// This will show queue statistics and discard information
291
```