0
# Audit Destinations
1

2
Pluggable audit destination implementations that enable sending audit events to various storage and messaging systems including HDFS, Solr, Kafka, ElasticSearch, CloudWatch, Log4j, and local files.
3

4
## Capabilities
5

6
### Base Audit Destination
7

8
Abstract base class that all audit destinations extend, providing lifecycle management and common functionality.
9

10
```java { .api }
11
/**
12
 * Base class for audit destinations
13
 */
14
public abstract class AuditDestination extends BaseAuditHandler {
15
    /**
16
     * Initialize the audit destination with configuration
17
     * @param props Properties configuration properties
18
     * @param basePropertyName String base property name for configuration
19
     */
20
    public abstract void init(Properties props, String basePropertyName);
21
    
22
    /**
23
     * Start the audit destination
24
     */
25
    public abstract void start();
26
    
27
    /**
28
     * Stop the audit destination
29
     */
30
    public abstract void stop();
31
    
32
    /**
33
     * Flush any pending audit events
34
     */
35
    public abstract void flush();
36
    
37
    /**
38
     * Wait for completion of pending operations
39
     */
40
    public void waitToComplete();
41
}
42
```
43

44
### HDFS Audit Destination
45

46
HDFS audit destination implementation supporting both JSON and ORC output formats with configurable file rotation and Kerberos authentication.
47

48
```java { .api }
49
/**
50
 * HDFS audit destination implementation
51
 */
52
public class HDFSAuditDestination extends AuditDestination {
53
    /**
54
     * Initialize HDFS destination with configuration
55
     * @param props Properties configuration properties
56
     * @param basePropertyName String base property name (e.g., "xasecure.audit.hdfs")
57
     */
58
    public void init(Properties props, String basePropertyName);
59
    
60
    /**
61
     * Log JSON-formatted audit events to HDFS
62
     * @param events Collection<String> JSON-formatted events
63
     */
64
    public void logJSON(Collection<String> events);
65
    
66
    /**
67
     * Log audit events from file to HDFS
68
     * @param file File containing audit events
69
     */
70
    public void logFile(File file);
71
    
72
    /**
73
     * Log structured audit events to HDFS
74
     * @param events Collection<AuditEventBase> structured events
75
     */
76
    public void log(Collection<AuditEventBase> events);
77
    
78
    /**
79
     * Start HDFS destination
80
     */
81
    public void start();
82
    
83
    /**
84
     * Stop HDFS destination
85
     */
86
    public void stop();
87
    
88
    /**
89
     * Flush pending events to HDFS
90
     */
91
    public void flush();
92
}
93
```
94

95
**HDFS Configuration Properties:**
96
- `xasecure.audit.hdfs.is.enabled`: Enable HDFS destination
97
- `xasecure.audit.hdfs.destination.directory`: HDFS directory path
98
- `xasecure.audit.hdfs.destination.file`: File name pattern
99
- `xasecure.audit.hdfs.destination.flush.interval.seconds`: Flush interval
100
- `xasecure.audit.hdfs.destination.rollover.interval.seconds`: File rollover interval
101

102
### Solr Audit Destination
103

104
Apache Solr audit destination with support for Solr Cloud, standalone Solr, and Kerberos authentication.
105

106
```java { .api }
107
/**
108
 * Apache Solr audit destination with Kerberos support
109
 */
110
public class SolrAuditDestination extends AuditDestination {
111
    // Configuration constants
112
    public static final String PROP_SOLR_URLS = "urls";
113
    public static final String PROP_SOLR_ZK = "zookeepers";
114
    public static final String PROP_SOLR_COLLECTION = "collection";
115
    public static final String DEFAULT_COLLECTION_NAME = "ranger_audits";
116
    
117
    /**
118
     * Initialize Solr destination with configuration
119
     * @param props Properties configuration properties
120
     * @param basePropertyName String base property name (e.g., "xasecure.audit.solr")
121
     */
122
    public void init(Properties props, String basePropertyName);
123
    
124
    /**
125
     * Stop Solr destination and clean up resources
126
     */
127
    public void stop();
128
    
129
    /**
130
     * Log structured audit events to Solr
131
     * @param events Collection<AuditEventBase> structured events
132
     */
133
    public void log(Collection<AuditEventBase> events);
134
    
135
    /**
136
     * Flush pending events to Solr
137
     */
138
    public void flush();
139
    
140
    /**
141
     * Check if destination supports asynchronous processing
142
     * @return boolean true if asynchronous
143
     */
144
    public boolean isAsync();
145
}
146
```
147

148
**Solr Configuration Properties:**
149
- `xasecure.audit.solr.is.enabled`: Enable Solr destination
150
- `xasecure.audit.solr.urls`: Solr server URLs (comma-separated)
151
- `xasecure.audit.solr.zookeepers`: ZooKeeper connection string for Solr Cloud
152
- `xasecure.audit.solr.collection`: Solr collection name
153

154
### Kafka Audit Provider
155

156
Apache Kafka audit provider supporting both synchronous and asynchronous message publishing with configurable partitioning and Kerberos authentication.
157

158
```java { .api }
159
/**
160
 * Apache Kafka audit provider
161
 */
162
public class KafkaAuditProvider extends BaseAuditHandler {
163
    /**
164
     * Initialize Kafka provider with configuration
165
     * @param props Properties configuration properties
166
     */
167
    public void init(Properties props);
168
    
169
    /**
170
     * Log single audit event to Kafka
171
     * @param event AuditEventBase event to log
172
     */
173
    public void log(AuditEventBase event);
174
    
175
    /**
176
     * Log collection of audit events to Kafka
177
     * @param events Collection<AuditEventBase> events to log
178
     */
179
    public void log(Collection<AuditEventBase> events);
180
    
181
    /**
182
     * Log JSON-formatted audit event to Kafka
183
     * @param event String JSON-formatted event
184
     */
185
    public void logJSON(String event);
186
    
187
    /**
188
     * Log collection of JSON-formatted events to Kafka
189
     * @param events Collection<String> JSON-formatted events
190
     */
191
    public void logJSON(Collection<String> events);
192
    
193
    /**
194
     * Start Kafka provider
195
     */
196
    public void start();
197
    
198
    /**
199
     * Stop Kafka provider
200
     */
201
    public void stop();
202
    
203
    /**
204
     * Flush pending messages to Kafka
205
     */
206
    public void flush();
207
    
208
    /**
209
     * Check if provider supports asynchronous processing
210
     * @return boolean true if asynchronous
211
     */
212
    public boolean isAsync();
213
}
214
```
215

216
**Kafka Configuration Properties:**
217
- `xasecure.audit.kafka.is.enabled`: Enable Kafka destination
218
- `xasecure.audit.kafka.broker_list`: Kafka broker list
219
- `xasecure.audit.kafka.topic_name`: Kafka topic name
220
- `xasecure.audit.kafka.producer.security.protocol`: Security protocol (PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL)
221

222
### ElasticSearch Audit Destination
223

224
ElasticSearch audit destination with support for ElasticSearch clusters, basic authentication, and Kerberos authentication using REST High Level Client.
225

226
```java { .api }
227
/**
228
 * ElasticSearch audit destination with authentication support
229
 */
230
public class ElasticSearchAuditDestination extends AuditDestination {
231
    // Configuration constants
232
    public static final String CONFIG_URLS = "urls";
233
    public static final String CONFIG_PORT = "port";
234
    public static final String CONFIG_USER = "user";
235
    public static final String CONFIG_PWRD = "password";
236
    public static final String CONFIG_PROTOCOL = "protocol";
237
    public static final String CONFIG_INDEX = "index";
238
    public static final String CONFIG_PREFIX = "ranger.audit.elasticsearch";
239
    public static final String DEFAULT_INDEX = "ranger_audits";
240
    
241
    /**
242
     * Initialize ElasticSearch destination with configuration
243
     * @param props Properties configuration properties
244
     * @param propPrefix String property prefix (e.g., "ranger.audit.elasticsearch")
245
     */
246
    public void init(Properties props, String propPrefix);
247
    
248
    /**
249
     * Stop ElasticSearch destination and cleanup resources
250
     */
251
    public void stop();
252
    
253
    /**
254
     * Log structured audit events to ElasticSearch using bulk indexing
255
     * @param events Collection<AuditEventBase> structured events
256
     * @return boolean true if all events logged successfully
257
     */
258
    public boolean log(Collection<AuditEventBase> events);
259
    
260
    /**
261
     * Flush pending events to ElasticSearch (no-op implementation)
262
     */
263
    public void flush();
264
    
265
    /**
266
     * Check if destination supports asynchronous processing
267
     * @return boolean true indicating async support
268
     */
269
    public boolean isAsync();
270
    
271
    /**
272
     * Get or create ElasticSearch REST client with connection management
273
     * @return RestHighLevelClient configured client instance
274
     */
275
    public synchronized RestHighLevelClient getClient();
276
    
277
    /**
278
     * Static factory method to create configured REST client builder
279
     * @param urls String comma-separated ElasticSearch hosts
280
     * @param protocol String connection protocol (http/https)
281
     * @param user String username for authentication
282
     * @param password String password or keytab file path
283
     * @param port int ElasticSearch port
284
     * @return RestClientBuilder configured client builder
285
     */
286
    public static RestClientBuilder getRestClientBuilder(String urls, String protocol, String user, String password, int port);
287
    
288
    /**
289
     * Convert audit event to ElasticSearch document
290
     * @param auditEvent AuthzAuditEvent event to convert
291
     * @return Map<String,Object> ElasticSearch document representation
292
     */
293
    public Map<String, Object> toDoc(AuthzAuditEvent auditEvent);
294
}
295
```
296

297
**ElasticSearch Configuration Properties:**
298
- `ranger.audit.elasticsearch.urls`: Comma-separated ElasticSearch hosts (required)
299
- `ranger.audit.elasticsearch.port`: ElasticSearch port (default: 9200)
300
- `ranger.audit.elasticsearch.protocol`: Connection protocol http/https (default: http)
301
- `ranger.audit.elasticsearch.user`: Username for authentication
302
- `ranger.audit.elasticsearch.password`: Password or keytab file path for Kerberos
303
- `ranger.audit.elasticsearch.index`: Target index name (default: ranger_audits)
304
```
305
306
### File Audit Destination
307

308
Local file audit destination for writing audit events to local filesystem files with configurable rotation.
309

310
```java { .api }
311
/**
312
 * Local file audit destination
313
 */
314
public class FileAuditDestination extends AuditDestination {
315
    /**
316
     * Initialize file destination with configuration
317
     * @param props Properties configuration properties
318
     * @param basePropertyName String base property name (e.g., "xasecure.audit.file")
319
     */
320
    public void init(Properties props, String basePropertyName);
321
    
322
    /**
323
     * Log JSON-formatted audit events to file
324
     * @param events Collection<String> JSON-formatted events
325
     */
326
    public void logJSON(Collection<String> events);
327
    
328
    /**
329
     * Log structured audit events to file
330
     * @param events Collection<AuditEventBase> structured events
331
     */
332
    public void log(Collection<AuditEventBase> events);
333
    
334
    /**
335
     * Start file destination
336
     */
337
    public void start();
338
    
339
    /**
340
     * Stop file destination
341
     */
342
    public void stop();
343
}
344
```
345

346
### Log4j Audit Destination
347

348
Log4j audit destination for sending audit events through the Log4j logging framework.
349

350
```java { .api }
351
/**
352
 * Log4j audit destination
353
 */
354
public class Log4JAuditDestination extends AuditDestination {
355
    /**
356
     * Initialize Log4j destination with configuration
357
     * @param props Properties configuration properties
358
     * @param basePropertyName String base property name
359
     */
360
    public void init(Properties props, String basePropertyName);
361
    
362
    /**
363
     * Log structured audit events via Log4j
364
     * @param events Collection<AuditEventBase> structured events
365
     */
366
    public void log(Collection<AuditEventBase> events);
367
}
368
```
369

370
### Amazon CloudWatch Audit Destination
371

372
Thread-safe Amazon CloudWatch audit destination for sending audit events to AWS CloudWatch Logs with automatic sequence token management and log stream creation.
373

374
```java { .api }
375
/**
376
 * Thread-safe Amazon CloudWatch audit destination
377
 */
378
@ThreadSafe
379
public class AmazonCloudWatchAuditDestination extends AuditDestination {
380
    // Configuration constants
381
    public static final String PROP_LOG_GROUP_NAME = "log_group";
382
    public static final String PROP_LOG_STREAM_PREFIX = "log_stream_prefix";
383
    public static final String CONFIG_PREFIX = "ranger.audit.amazon_cloudwatch";
384
    public static final String PROP_REGION = "region";
385
    
386
    /**
387
     * Initialize CloudWatch destination with AWS configuration
388
     * @param props Properties configuration properties
389
     * @param propPrefix String property prefix (e.g., "ranger.audit.amazon_cloudwatch")
390
     */
391
    public void init(Properties props, String propPrefix);
392
    
393
    /**
394
     * Stop CloudWatch destination and log final status
395
     */
396
    public void stop();
397
    
398
    /**
399
     * Log structured audit events to CloudWatch Logs (thread-safe)
400
     * @param collection Collection<AuditEventBase> audit events to log
401
     * @return boolean true if all events logged successfully
402
     */
403
    public synchronized boolean log(Collection<AuditEventBase> collection);
404
    
405
    /**
406
     * Flush pending events to CloudWatch (no-op implementation)
407
     */
408
    public void flush();
409
    
410
    /**
411
     * Convert audit events to CloudWatch InputLogEvent format
412
     * @param collection Collection<AuditEventBase> events to convert
413
     * @return Collection<InputLogEvent> sorted by timestamp
414
     */
415
    public static Collection<InputLogEvent> toInputLogEvent(Collection<AuditEventBase> collection);
416
}
417
```
418

419
**CloudWatch Configuration Properties:**
420
- `ranger.audit.amazon_cloudwatch.log_group`: CloudWatch Log Group name (default: "ranger_audits")
421
- `ranger.audit.amazon_cloudwatch.log_stream_prefix`: Log stream name prefix (required)
422
- `ranger.audit.amazon_cloudwatch.region`: AWS region (optional, uses AWS default if not specified)
423

424
**Important CloudWatch Features:**
425
- Thread-safe implementation with synchronized log method
426
- Automatic sequence token management for CloudWatch API
427
- Automatic log stream creation if not exists
428
- Events sorted by timestamp (CloudWatch requirement)
429
- Retry logic for InvalidSequenceTokenException
430
- Uses AWS SDK default credential chain for authentication
431
```
432
433
**Usage Examples:**
434

435
```java
436
import org.apache.ranger.audit.destination.*;
437
import org.apache.ranger.audit.model.AuthzAuditEvent;
438

439
// HDFS destination configuration
440
Properties hdfsProps = new Properties();
441
hdfsProps.setProperty("xasecure.audit.hdfs.is.enabled", "true");
442
hdfsProps.setProperty("xasecure.audit.hdfs.destination.directory", "/ranger/audit/%app-type%/%time:yyyyMMdd%");
443
hdfsProps.setProperty("xasecure.audit.hdfs.destination.file", "%hostname%-audit.log");
444

445
HDFSAuditDestination hdfsDestination = new HDFSAuditDestination();
446
hdfsDestination.init(hdfsProps, "xasecure.audit.hdfs");
447
hdfsDestination.start();
448

449
// Solr destination configuration
450
Properties solrProps = new Properties();
451
solrProps.setProperty("xasecure.audit.solr.is.enabled", "true");
452
solrProps.setProperty("xasecure.audit.solr.urls", "http://solr1:8983/solr,http://solr2:8983/solr");
453
solrProps.setProperty("xasecure.audit.solr.collection", "ranger_audits");
454

455
SolrAuditDestination solrDestination = new SolrAuditDestination();
456
solrDestination.init(solrProps, "xasecure.audit.solr");
457

458
// Log events to multiple destinations
459
AuthzAuditEvent event = new AuthzAuditEvent();
460
// ... set event properties ...
461

462
List<AuditEventBase> events = Arrays.asList(event);
463
hdfsDestination.log(events);
464
solrDestination.log(events);
465
```