CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/maven-org-apache-ranger--ranger-plugins-audit

Apache Ranger Audit Plugin Framework providing centralized audit logging capabilities for Apache Ranger security plugins across various big data components.

—

Pending
Overview
Eval results
Files

audit-destinations.mddocs/

Audit Destinations

Pluggable audit destination implementations that enable sending audit events to various storage and messaging systems including HDFS, Solr, Kafka, ElasticSearch, CloudWatch, Log4j, and local files.

Capabilities

Base Audit Destination

Abstract base class that all audit destinations extend, providing lifecycle management and common functionality.

/**
 * Base class for audit destinations
 */
public abstract class AuditDestination extends BaseAuditHandler {
    /**
     * Initialize the audit destination with configuration
     * @param props Properties configuration properties
     * @param basePropertyName String base property name for configuration
     */
    public abstract void init(Properties props, String basePropertyName);
    
    /**
     * Start the audit destination
     */
    public abstract void start();
    
    /**
     * Stop the audit destination
     */
    public abstract void stop();
    
    /**
     * Flush any pending audit events
     */
    public abstract void flush();
    
    /**
     * Wait for completion of pending operations
     */
    public void waitToComplete();
}

HDFS Audit Destination

HDFS audit destination implementation supporting both JSON and ORC output formats with configurable file rotation and Kerberos authentication.

/**
 * HDFS audit destination implementation
 */
public class HDFSAuditDestination extends AuditDestination {
    /**
     * Initialize HDFS destination with configuration
     * @param props Properties configuration properties
     * @param basePropertyName String base property name (e.g., "xasecure.audit.hdfs")
     */
    public void init(Properties props, String basePropertyName);
    
    /**
     * Log JSON-formatted audit events to HDFS
     * @param events Collection<String> JSON-formatted events
     */
    public void logJSON(Collection<String> events);
    
    /**
     * Log audit events from file to HDFS
     * @param file File containing audit events
     */
    public void logFile(File file);
    
    /**
     * Log structured audit events to HDFS
     * @param events Collection<AuditEventBase> structured events
     */
    public void log(Collection<AuditEventBase> events);
    
    /**
     * Start HDFS destination
     */
    public void start();
    
    /**
     * Stop HDFS destination
     */
    public void stop();
    
    /**
     * Flush pending events to HDFS
     */
    public void flush();
}

HDFS Configuration Properties:

  • xasecure.audit.hdfs.is.enabled: Enable HDFS destination
  • xasecure.audit.hdfs.destination.directory: HDFS directory path
  • xasecure.audit.hdfs.destination.file: File name pattern
  • xasecure.audit.hdfs.destination.flush.interval.seconds: Flush interval
  • xasecure.audit.hdfs.destination.rollover.interval.seconds: File rollover interval

Solr Audit Destination

Apache Solr audit destination with support for Solr Cloud, standalone Solr, and Kerberos authentication.

/**
 * Apache Solr audit destination with Kerberos support
 */
public class SolrAuditDestination extends AuditDestination {
    // Configuration constants
    public static final String PROP_SOLR_URLS = "urls";
    public static final String PROP_SOLR_ZK = "zookeepers";
    public static final String PROP_SOLR_COLLECTION = "collection";
    public static final String DEFAULT_COLLECTION_NAME = "ranger_audits";
    
    /**
     * Initialize Solr destination with configuration
     * @param props Properties configuration properties
     * @param basePropertyName String base property name (e.g., "xasecure.audit.solr")
     */
    public void init(Properties props, String basePropertyName);
    
    /**
     * Stop Solr destination and clean up resources
     */
    public void stop();
    
    /**
     * Log structured audit events to Solr
     * @param events Collection<AuditEventBase> structured events
     */
    public void log(Collection<AuditEventBase> events);
    
    /**
     * Flush pending events to Solr
     */
    public void flush();
    
    /**
     * Check if destination supports asynchronous processing
     * @return boolean true if asynchronous
     */
    public boolean isAsync();
}

Solr Configuration Properties:

  • xasecure.audit.solr.is.enabled: Enable Solr destination
  • xasecure.audit.solr.urls: Solr server URLs (comma-separated)
  • xasecure.audit.solr.zookeepers: ZooKeeper connection string for Solr Cloud
  • xasecure.audit.solr.collection: Solr collection name

Kafka Audit Provider

Apache Kafka audit provider supporting both synchronous and asynchronous message publishing with configurable partitioning and Kerberos authentication.

/**
 * Apache Kafka audit provider
 */
public class KafkaAuditProvider extends BaseAuditHandler {
    /**
     * Initialize Kafka provider with configuration
     * @param props Properties configuration properties
     */
    public void init(Properties props);
    
    /**
     * Log single audit event to Kafka
     * @param event AuditEventBase event to log
     */
    public void log(AuditEventBase event);
    
    /**
     * Log collection of audit events to Kafka
     * @param events Collection<AuditEventBase> events to log
     */
    public void log(Collection<AuditEventBase> events);
    
    /**
     * Log JSON-formatted audit event to Kafka
     * @param event String JSON-formatted event
     */
    public void logJSON(String event);
    
    /**
     * Log collection of JSON-formatted events to Kafka
     * @param events Collection<String> JSON-formatted events
     */
    public void logJSON(Collection<String> events);
    
    /**
     * Start Kafka provider
     */
    public void start();
    
    /**
     * Stop Kafka provider
     */
    public void stop();
    
    /**
     * Flush pending messages to Kafka
     */
    public void flush();
    
    /**
     * Check if provider supports asynchronous processing
     * @return boolean true if asynchronous
     */
    public boolean isAsync();
}

Kafka Configuration Properties:

  • xasecure.audit.kafka.is.enabled: Enable Kafka destination
  • xasecure.audit.kafka.broker_list: Kafka broker list
  • xasecure.audit.kafka.topic_name: Kafka topic name
  • xasecure.audit.kafka.producer.security.protocol: Security protocol (PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL)

ElasticSearch Audit Destination

ElasticSearch audit destination with support for ElasticSearch clusters, basic authentication, and Kerberos authentication using REST High Level Client.

/**
 * ElasticSearch audit destination with authentication support
 */
public class ElasticSearchAuditDestination extends AuditDestination {
    // Configuration constants
    public static final String CONFIG_URLS = "urls";
    public static final String CONFIG_PORT = "port";
    public static final String CONFIG_USER = "user";
    public static final String CONFIG_PWRD = "password";
    public static final String CONFIG_PROTOCOL = "protocol";
    public static final String CONFIG_INDEX = "index";
    public static final String CONFIG_PREFIX = "ranger.audit.elasticsearch";
    public static final String DEFAULT_INDEX = "ranger_audits";
    
    /**
     * Initialize ElasticSearch destination with configuration
     * @param props Properties configuration properties
     * @param propPrefix String property prefix (e.g., "ranger.audit.elasticsearch")
     */
    public void init(Properties props, String propPrefix);
    
    /**
     * Stop ElasticSearch destination and cleanup resources
     */
    public void stop();
    
    /**
     * Log structured audit events to ElasticSearch using bulk indexing
     * @param events Collection<AuditEventBase> structured events
     * @return boolean true if all events logged successfully
     */
    public boolean log(Collection<AuditEventBase> events);
    
    /**
     * Flush pending events to ElasticSearch (no-op implementation)
     */
    public void flush();
    
    /**
     * Check if destination supports asynchronous processing
     * @return boolean true indicating async support
     */
    public boolean isAsync();
    
    /**
     * Get or create ElasticSearch REST client with connection management
     * @return RestHighLevelClient configured client instance
     */
    public synchronized RestHighLevelClient getClient();
    
    /**
     * Static factory method to create configured REST client builder
     * @param urls String comma-separated ElasticSearch hosts
     * @param protocol String connection protocol (http/https)
     * @param user String username for authentication
     * @param password String password or keytab file path
     * @param port int ElasticSearch port
     * @return RestClientBuilder configured client builder
     */
    public static RestClientBuilder getRestClientBuilder(String urls, String protocol, String user, String password, int port);
    
    /**
     * Convert audit event to ElasticSearch document
     * @param auditEvent AuthzAuditEvent event to convert
     * @return Map<String,Object> ElasticSearch document representation
     */
    public Map<String, Object> toDoc(AuthzAuditEvent auditEvent);
}

ElasticSearch Configuration Properties:

  • ranger.audit.elasticsearch.urls: Comma-separated ElasticSearch hosts (required)
  • ranger.audit.elasticsearch.port: ElasticSearch port (default: 9200)
  • ranger.audit.elasticsearch.protocol: Connection protocol http/https (default: http)
  • ranger.audit.elasticsearch.user: Username for authentication
  • ranger.audit.elasticsearch.password: Password or keytab file path for Kerberos
  • ranger.audit.elasticsearch.index: Target index name (default: ranger_audits)
### File Audit Destination

Local file audit destination for writing audit events to local filesystem files with configurable rotation.

```java { .api }
/**
 * Local file audit destination
 */
public class FileAuditDestination extends AuditDestination {
    /**
     * Initialize file destination with configuration
     * @param props Properties configuration properties
     * @param basePropertyName String base property name (e.g., "xasecure.audit.file")
     */
    public void init(Properties props, String basePropertyName);
    
    /**
     * Log JSON-formatted audit events to file
     * @param events Collection<String> JSON-formatted events
     */
    public void logJSON(Collection<String> events);
    
    /**
     * Log structured audit events to file
     * @param events Collection<AuditEventBase> structured events
     */
    public void log(Collection<AuditEventBase> events);
    
    /**
     * Start file destination
     */
    public void start();
    
    /**
     * Stop file destination
     */
    public void stop();
}

Log4j Audit Destination

Log4j audit destination for sending audit events through the Log4j logging framework.

/**
 * Log4j audit destination
 */
public class Log4JAuditDestination extends AuditDestination {
    /**
     * Initialize Log4j destination with configuration
     * @param props Properties configuration properties
     * @param basePropertyName String base property name
     */
    public void init(Properties props, String basePropertyName);
    
    /**
     * Log structured audit events via Log4j
     * @param events Collection<AuditEventBase> structured events
     */
    public void log(Collection<AuditEventBase> events);
}

Amazon CloudWatch Audit Destination

Thread-safe Amazon CloudWatch audit destination for sending audit events to AWS CloudWatch Logs with automatic sequence token management and log stream creation.

/**
 * Thread-safe Amazon CloudWatch audit destination
 */
@ThreadSafe
public class AmazonCloudWatchAuditDestination extends AuditDestination {
    // Configuration constants
    public static final String PROP_LOG_GROUP_NAME = "log_group";
    public static final String PROP_LOG_STREAM_PREFIX = "log_stream_prefix";
    public static final String CONFIG_PREFIX = "ranger.audit.amazon_cloudwatch";
    public static final String PROP_REGION = "region";
    
    /**
     * Initialize CloudWatch destination with AWS configuration
     * @param props Properties configuration properties
     * @param propPrefix String property prefix (e.g., "ranger.audit.amazon_cloudwatch")
     */
    public void init(Properties props, String propPrefix);
    
    /**
     * Stop CloudWatch destination and log final status
     */
    public void stop();
    
    /**
     * Log structured audit events to CloudWatch Logs (thread-safe)
     * @param collection Collection<AuditEventBase> audit events to log
     * @return boolean true if all events logged successfully
     */
    public synchronized boolean log(Collection<AuditEventBase> collection);
    
    /**
     * Flush pending events to CloudWatch (no-op implementation)
     */
    public void flush();
    
    /**
     * Convert audit events to CloudWatch InputLogEvent format
     * @param collection Collection<AuditEventBase> events to convert
     * @return Collection<InputLogEvent> sorted by timestamp
     */
    public static Collection<InputLogEvent> toInputLogEvent(Collection<AuditEventBase> collection);
}

CloudWatch Configuration Properties:

  • ranger.audit.amazon_cloudwatch.log_group: CloudWatch Log Group name (default: "ranger_audits")
  • ranger.audit.amazon_cloudwatch.log_stream_prefix: Log stream name prefix (required)
  • ranger.audit.amazon_cloudwatch.region: AWS region (optional, uses AWS default if not specified)

Important CloudWatch Features:

  • Thread-safe implementation with synchronized log method
  • Automatic sequence token management for CloudWatch API
  • Automatic log stream creation if not exists
  • Events sorted by timestamp (CloudWatch requirement)
  • Retry logic for InvalidSequenceTokenException
  • Uses AWS SDK default credential chain for authentication
**Usage Examples:**

```java
import org.apache.ranger.audit.destination.*;
import org.apache.ranger.audit.model.AuthzAuditEvent;

// HDFS destination configuration
Properties hdfsProps = new Properties();
hdfsProps.setProperty("xasecure.audit.hdfs.is.enabled", "true");
hdfsProps.setProperty("xasecure.audit.hdfs.destination.directory", "/ranger/audit/%app-type%/%time:yyyyMMdd%");
hdfsProps.setProperty("xasecure.audit.hdfs.destination.file", "%hostname%-audit.log");

HDFSAuditDestination hdfsDestination = new HDFSAuditDestination();
hdfsDestination.init(hdfsProps, "xasecure.audit.hdfs");
hdfsDestination.start();

// Solr destination configuration
Properties solrProps = new Properties();
solrProps.setProperty("xasecure.audit.solr.is.enabled", "true");
solrProps.setProperty("xasecure.audit.solr.urls", "http://solr1:8983/solr,http://solr2:8983/solr");
solrProps.setProperty("xasecure.audit.solr.collection", "ranger_audits");

SolrAuditDestination solrDestination = new SolrAuditDestination();
solrDestination.init(solrProps, "xasecure.audit.solr");

// Log events to multiple destinations
AuthzAuditEvent event = new AuthzAuditEvent();
// ... set event properties ...

List<AuditEventBase> events = Arrays.asList(event);
hdfsDestination.log(events);
solrDestination.log(events);

Install with Tessl CLI

npx tessl i tessl/maven-org-apache-ranger--ranger-plugins-audit

docs

audit-destinations.md

core-framework.md

index.md

queue-async.md

writers-formats.md

tile.json