CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/maven-io-cdap-cdap--cdap-watchdog

CDAP Watchdog provides comprehensive metrics collection, querying, and logging services for the CDAP platform

Pending
Overview
Eval results
Files

logging-context.mddocs/

Logging Context System

Context classes and utilities for organizing logs by CDAP program types, providing structured logging with consistent tagging, filtering capabilities, and program-specific log organization across the CDAP platform.

Capabilities

LoggingContextHelper

Utility class for creating and manipulating logging contexts for different program types, providing the primary factory methods for context creation.

/**
 * Creates and manipulates logging contexts for different program types
 * Primary factory for creating appropriate logging contexts based on program type
 */
public final class LoggingContextHelper {
    /**
     * Create logging context for a CDAP program
     * @param namespaceId Namespace identifier
     * @param appId Application identifier  
     * @param entityId Program/entity identifier
     * @param programType Type of CDAP program (SERVICE, WORKFLOW, MAPREDUCE, etc.)
     * @return LoggingContext appropriate for the specified program type
     */
    public static LoggingContext getLoggingContext(String namespaceId, String appId, String entityId, ProgramType programType);
    
    /**
     * Create logging context with run ID for a specific program execution
     * @param namespaceId Namespace identifier
     * @param appId Application identifier
     * @param entityId Program/entity identifier  
     * @param programType Type of CDAP program
     * @param runId Unique identifier for this program run
     * @return LoggingContext with run-specific information
     */
    public static LoggingContext getLoggingContextWithRunId(String namespaceId, String appId, String entityId, ProgramType programType, String runId);
    
    /**
     * Create filter from logging context
     * @param loggingContext Context to create filter from
     * @return Filter object for log querying based on context
     */
    public static Filter createFilter(LoggingContext loggingContext);
    
    /**
     * Extract metrics tags from logging context
     * @param loggingContext Context to extract tags from  
     * @return Map of tag names to tag values for metrics tagging
     */
    public static Map<String, String> getMetricsTags(LoggingContext loggingContext);
}

Usage Examples:

import io.cdap.cdap.logging.context.LoggingContextHelper;
import io.cdap.cdap.logging.context.LoggingContext;
import io.cdap.cdap.common.app.ProgramType;

// Create context for a service
LoggingContext serviceContext = LoggingContextHelper.getLoggingContext(
    "myNamespace",           // Namespace
    "myApplication",         // Application  
    "myService",            // Service name
    ProgramType.SERVICE     // Program type
);

// Create context for a specific workflow run
LoggingContext workflowRunContext = LoggingContextHelper.getLoggingContextWithRunId(
    "myNamespace",
    "myApplication", 
    "myWorkflow",
    ProgramType.WORKFLOW,
    "run-12345"             // Specific run ID
);

// Create filter for log queries
Filter logFilter = LoggingContextHelper.createFilter(serviceContext);

// Extract metrics tags for correlation
Map<String, String> metricsTags = LoggingContextHelper.getMetricsTags(serviceContext);

Base Application Context

Base context class for applications, providing common functionality for all program types.

/**
 * Base context for applications
 * Provides common logging context functionality for all CDAP programs
 */
public class ApplicationLoggingContext extends AbstractLoggingContext {
    /**
     * Create application logging context
     * @param namespaceId Namespace identifier
     * @param applicationId Application identifier
     * @param runId Run identifier (optional)
     */
    public ApplicationLoggingContext(String namespaceId, String applicationId, String runId);
    
    /**
     * Get the log partition for this context
     * Used for organizing logs in storage systems
     * @return String representing the log partition
     */
    public String getLogPartition();
}

Service Context

Logging context for user-defined services with handler-specific tagging.

/**
 * Context for user-defined services
 * Provides logging context for CDAP user services with handler support
 */
public class UserServiceLoggingContext extends ApplicationLoggingContext {
    /** Tag name for user service ID */
    public static final String TAG_USER_SERVICE_ID = ".userserviceid";
    
    /** Tag name for handler ID within the service */
    public static final String TAG_HANDLER_ID = ".userhandlerid";
    
    /**
     * Create user service logging context
     * @param namespaceId Namespace identifier
     * @param applicationId Application identifier
     * @param serviceId User service identifier
     * @param runId Run identifier (optional)
     */
    public UserServiceLoggingContext(String namespaceId, String applicationId, String serviceId, String runId);
    
    /**
     * Get the log partition for this service context
     * @return String representing the log partition for this service
     */
    public String getLogPartition();
}

Workflow Context

Logging context for workflow programs with workflow-specific tagging.

/**
 * Context for workflow programs  
 * Provides logging context for CDAP workflow executions
 */
public class WorkflowLoggingContext extends ApplicationLoggingContext {
    /** Tag name for workflow ID */
    public static final String TAG_WORKFLOW_ID = ".workflowid";
    
    /**
     * Create workflow logging context
     * @param namespaceId Namespace identifier
     * @param applicationId Application identifier
     * @param workflowId Workflow identifier
     * @param runId Run identifier (optional)
     */
    public WorkflowLoggingContext(String namespaceId, String applicationId, String workflowId, String runId);
    
    /**
     * Get the log partition for this workflow context
     * @return String representing the log partition for this workflow
     */
    public String getLogPartition();
}

Workflow Program Context

Specialized context for programs running within workflows.

/**
 * Context for programs within workflows
 * Provides logging context for programs executed as part of a workflow
 */
public class WorkflowProgramLoggingContext extends WorkflowLoggingContext {
    /**
     * Create workflow program logging context
     * @param namespaceId Namespace identifier
     * @param applicationId Application identifier  
     * @param workflowId Workflow identifier
     * @param programType Type of program within workflow
     * @param programId Program identifier within workflow
     * @param runId Run identifier (optional)
     */
    public WorkflowProgramLoggingContext(String namespaceId, String applicationId, String workflowId, ProgramType programType, String programId, String runId);
    
    /**
     * Get the log partition for this workflow program context
     * @return String representing the log partition for this workflow program
     */
    public String getLogPartition();
}

MapReduce Context

Logging context for MapReduce programs with job-specific tagging.

/**
 * Context for MapReduce programs
 * Provides logging context for CDAP MapReduce job executions
 */
public class MapReduceLoggingContext extends ApplicationLoggingContext {
    /** Tag name for MapReduce job ID */
    public static final String TAG_MAP_REDUCE_JOB_ID = ".mapreducejobid";
    
    /**
     * Create MapReduce logging context
     * @param namespaceId Namespace identifier
     * @param applicationId Application identifier
     * @param mapReduceId MapReduce program identifier
     * @param runId Run identifier (optional)
     */
    public MapReduceLoggingContext(String namespaceId, String applicationId, String mapReduceId, String runId);
    
    /**
     * Get the log partition for this MapReduce context
     * @return String representing the log partition for this MapReduce job
     */
    public String getLogPartition();
}

Spark Context

Logging context for Spark programs with job-specific tagging.

/**
 * Context for Spark programs
 * Provides logging context for CDAP Spark job executions  
 */
public class SparkLoggingContext extends ApplicationLoggingContext {
    /** Tag name for Spark job ID */
    public static final String TAG_SPARK_JOB_ID = ".sparkjobid";
    
    /**
     * Create Spark logging context
     * @param namespaceId Namespace identifier
     * @param applicationId Application identifier
     * @param sparkId Spark program identifier
     * @param runId Run identifier (optional)
     */
    public SparkLoggingContext(String namespaceId, String applicationId, String sparkId, String runId);
    
    /**
     * Get the log partition for this Spark context
     * @return String representing the log partition for this Spark job
     */
    public String getLogPartition();
}

Worker Context

Logging context for worker programs with worker-specific tagging.

/**
 * Context for worker programs
 * Provides logging context for CDAP worker program executions
 */
public class WorkerLoggingContext extends ApplicationLoggingContext {
    /** Tag name for worker ID */
    public static final String TAG_WORKER_ID = ".workerid";
    
    /**
     * Create worker logging context
     * @param namespaceId Namespace identifier
     * @param applicationId Application identifier
     * @param workerId Worker program identifier
     * @param runId Run identifier (optional)
     */
    public WorkerLoggingContext(String namespaceId, String applicationId, String workerId, String runId);
    
    /**
     * Get the log partition for this worker context
     * @return String representing the log partition for this worker
     */
    public String getLogPartition();
}

Context Usage Examples:

import io.cdap.cdap.logging.context.*;
import io.cdap.cdap.common.app.ProgramType;

// Service context with handler
UserServiceLoggingContext serviceContext = new UserServiceLoggingContext(
    "production",           // Namespace
    "data-pipeline",       // Application
    "http-service",        // Service ID  
    "run-67890"           // Run ID
);

// MapReduce context
MapReduceLoggingContext mrContext = new MapReduceLoggingContext(
    "analytics", 
    "batch-processor",
    "daily-aggregation",
    "run-11111"
);

// Workflow with embedded program
WorkflowLoggingContext workflowContext = new WorkflowLoggingContext(
    "etl",
    "data-ingestion", 
    "nightly-workflow",
    "run-22222"
);

WorkflowProgramLoggingContext workflowProgramContext = new WorkflowProgramLoggingContext(
    "etl",
    "data-ingestion",
    "nightly-workflow", 
    ProgramType.MAPREDUCE,
    "transform-step",
    "run-22222"
);

// Extract partition information for storage
String servicePartition = serviceContext.getLogPartition();
String mrPartition = mrContext.getLogPartition();

Context Tag Constants

Each logging context class defines public constants for tag names used in log organization and filtering:

// User Service Context Tags
public static final String TAG_USER_SERVICE_ID = ".userserviceid";
public static final String TAG_HANDLER_ID = ".userhandlerid";

// Workflow Context Tags  
public static final String TAG_WORKFLOW_ID = ".workflowid";

// MapReduce Context Tags
public static final String TAG_MAP_REDUCE_JOB_ID = ".mapreducejobid";

// Spark Context Tags
public static final String TAG_SPARK_JOB_ID = ".sparkjobid";

// Worker Context Tags
public static final String TAG_WORKER_ID = ".workerid";

These tag constants are used internally for consistent log tagging and can be used by external code for log filtering and querying operations.

Install with Tessl CLI

npx tessl i tessl/maven-io-cdap-cdap--cdap-watchdog

docs

index.md

log-buffer.md

logging-context.md

logging-service.md

metrics-collection.md

metrics-processing.md

metrics-query.md

tile.json