0
# Apache Ranger Audit Plugin Framework
1

2
Apache Ranger Audit Plugin Framework provides centralized audit logging capabilities for Apache Ranger security plugins across various big data components (HDFS, Hive, HBase, Knox, etc.). It implements a configurable audit queue system with support for batching, buffering, and multiple destination types including HDFS, Solr, Elasticsearch, Kafka, CloudWatch, and Log4j.
3

4
## Package Information
5

6
- **Package Name**: org.apache.ranger:ranger-plugins-audit
7
- **Package Type**: maven
8
- **Language**: Java
9
- **Installation**: Add to Maven dependencies: `<groupId>org.apache.ranger</groupId><artifactId>ranger-plugins-audit</artifactId><version>2.7.0</version>`
10

11
## Core Imports
12

13
```java
14
import org.apache.ranger.audit.provider.AuditProviderFactory;
15
import org.apache.ranger.audit.provider.AuditHandler;
16
import org.apache.ranger.audit.model.AuthzAuditEvent;
17
```
18

19
## Basic Usage
20

21
```java
22
import org.apache.ranger.audit.provider.AuditProviderFactory;
23
import org.apache.ranger.audit.provider.AuditHandler;
24
import org.apache.ranger.audit.model.AuthzAuditEvent;
25
import java.util.Properties;
26

27
// Initialize audit framework
28
Properties auditProps = new Properties();
29
auditProps.setProperty("xasecure.audit.is.enabled", "true");
30
auditProps.setProperty("xasecure.audit.hdfs.is.enabled", "true");
31
auditProps.setProperty("xasecure.audit.hdfs.destination.directory", "/ranger/audit");
32

33
AuditProviderFactory factory = AuditProviderFactory.getInstance();
34
factory.init(auditProps, "myapp");
35

36
// Get audit provider
37
AuditHandler auditProvider = factory.getAuditProvider();
38

39
// Create and log audit event
40
AuthzAuditEvent auditEvent = new AuthzAuditEvent();
41
auditEvent.setRepositoryName("myservice");
42
auditEvent.setRepositoryType(EnumRepositoryType.HDFS);
43
auditEvent.setUser("john.doe");
44
auditEvent.setAccessType("read");
45
auditEvent.setResourceType("path");
46
auditEvent.setResourcePath("/data/sensitive");
47
auditEvent.setAccessResult(1); // ALLOWED
48
auditEvent.setEventTime(new Date());
49

50
// Log the audit event
51
auditProvider.log(auditEvent);
52

53
// Shutdown
54
factory.shutdown();
55
```
56

57
## Architecture
58

59
The framework is organized as a multi-module Maven project with a layered architecture:
60

61
- **Core Framework**: Base classes, interfaces, and factory patterns for audit handling
62
- **Event Model**: Rich audit event structures with extensive metadata support
63
- **Queue System**: Asynchronous processing with configurable batching and file spooling
64
- **Destination Plugins**: Multiple audit destinations (HDFS, Solr, Kafka, ElasticSearch, CloudWatch, Log4j, local files)
65
- **Writer Implementations**: Support for JSON and ORC file formats
66
- **Security Integration**: Kerberos authentication support across destinations
67

68
This design enables reliable audit trails across the entire Hadoop ecosystem through standardized audit event processing with configurable reliability, performance, and destination options.
69

70
## Capabilities
71

72
### Core Audit Framework
73

74
Primary audit framework components including factories, handlers, and event models that form the foundation of the audit system.
75

76
```java { .api }
77
// Main factory for audit providers
78
public class AuditProviderFactory {
79
    public static AuditProviderFactory getInstance();
80
    public void init(Properties props, String appType);
81
    public AuditHandler getAuditProvider();
82
    public void shutdown();
83
}
84

85
// Primary interface for audit handlers
86
public interface AuditHandler {
87
    public boolean log(AuditEventBase event);
88
    public boolean log(Collection<AuditEventBase> events);
89
    public boolean logJSON(String event);
90
    public boolean logJSON(Collection<String> events);
91
    public boolean logFile(File file);
92
    public void init(Properties props);
93
    public void init(Properties props, String basePropertyName);
94
    public void start();
95
    public void stop();
96
    public void waitToComplete();
97
    public void waitToComplete(long timeout);
98
    public String getName();
99
    public void flush();
100
}
101

102
// Primary audit event model
103
public class AuthzAuditEvent extends AuditEventBase {
104
    // Comprehensive getters/setters for audit event data
105
}
106
```
107

108
[Core Framework](./core-framework.md)
109

110
### Audit Destinations
111

112
Pluggable audit destination implementations for sending audit events to various storage and messaging systems.
113

114
```java { .api }
115
// Base class for audit destinations
116
public abstract class AuditDestination extends BaseAuditHandler {
117
    public abstract void init(Properties props, String basePropertyName);
118
    public abstract void start();
119
    public abstract void stop();
120
    public abstract void flush();
121
}
122

123
// HDFS audit destination
124
public class HDFSAuditDestination extends AuditDestination {
125
    public void init(Properties props, String basePropertyName);
126
    public void logJSON(Collection<String> events);
127
    public void logFile(File file);
128
}
129

130
// Solr audit destination  
131
public class SolrAuditDestination extends AuditDestination {
132
    public void init(Properties props, String basePropertyName);
133
    public void log(Collection<AuditEventBase> events);
134
}
135
```
136

137
[Audit Destinations](./audit-destinations.md)
138

139
### Queue and Async Processing
140

141
Asynchronous audit processing with configurable queues, batching, and file spooling for reliability.
142

143
```java { .api }
144
// Asynchronous audit provider
145
public class AsyncAuditProvider extends BaseAuditHandler {
146
    public AsyncAuditProvider(String name, int maxQueueSize, int maxBatchInterval);
147
    public void init(Properties props);
148
    public void log(AuditEventBase event);
149
    public void start();
150
    public void stop();
151
    public void waitToComplete();
152
}
153

154
// Base class for audit queues
155
public abstract class AuditQueue extends AuditDestination {
156
    // Queue configuration and drain management methods
157
}
158

159
// Multi-destination audit provider
160
public class MultiDestAuditProvider extends BaseAuditHandler {
161
    public void addAuditProvider(AuditHandler provider);
162
    public void addAuditProviders(List<AuditHandler> providers);
163
}
164
```
165

166
[Queue and Async Processing](./queue-async.md)
167

168
### File Writers and Formats
169

170
Audit file writers supporting multiple output formats including JSON and ORC.
171

172
```java { .api }
173
// Interface for audit file writers
174
public interface RangerAuditWriter {
175
    public void init(Properties props, String basePropertyName, String auditProviderName, Map<String,String> auditConfigs);
176
    public void log(Collection<String> events);
177
    public void logFile(File file);
178
    public void start();
179
    public void stop();
180
    public void flush();
181
}
182

183
// JSON audit writer for HDFS
184
public class RangerJSONAuditWriter extends AbstractRangerAuditWriter {
185
    public void init(Properties props, String basePropertyName, String auditProviderName, Map<String,String> auditConfigs);
186
    public void log(Collection<String> events);
187
}
188

189
// ORC audit writer
190
public class RangerORCAuditWriter extends AbstractRangerAuditWriter {
191
    public void init(Properties props, String basePropertyName, String auditProviderName, Map<String,String> auditConfigs);
192
    public void log(Collection<String> events);
193
}
194
```
195

196
[Writers and Formats](./writers-formats.md)
197

198
## Types
199

200
```java { .api }
201
// Base abstract class for all audit events
202
public abstract class AuditEventBase {
203
    public abstract String getEventKey();
204
    public Date getEventTime();
205
    public void setEventCount(long eventCount);
206
    public void setEventDurationMS(long eventDurationMS);
207
}
208

209
// Repository type constants
210
public final class EnumRepositoryType {
211
    public static final int HDFS = 1;
212
    public static final int HBASE = 2;
213
    public static final int HIVE = 3;
214
    public static final int XAAGENT = 4;
215
    public static final int KNOX = 5;
216
    public static final int STORM = 6;
217
}
218

219
// Spool file status enumeration
220
public enum SPOOL_FILE_STATUS {
221
    pending, write_inprogress, read_inprogress, done
222
}
223

224
// Audit exception type
225
public class AuditMessageException extends Exception {
226
    public AuditMessageException(String message);
227
    public AuditMessageException(String message, Throwable cause);
228
}
229
```