CDAP Watchdog provides comprehensive metrics collection, querying, and logging services for the CDAP platform
—
Context classes and utilities for organizing logs by CDAP program types, providing structured logging with consistent tagging, filtering capabilities, and program-specific log organization across the CDAP platform.
Utility class for creating and manipulating logging contexts for different program types, providing the primary factory methods for context creation.
/**
* Creates and manipulates logging contexts for different program types
* Primary factory for creating appropriate logging contexts based on program type
*/
public final class LoggingContextHelper {
/**
* Create logging context for a CDAP program
* @param namespaceId Namespace identifier
* @param appId Application identifier
* @param entityId Program/entity identifier
* @param programType Type of CDAP program (SERVICE, WORKFLOW, MAPREDUCE, etc.)
* @return LoggingContext appropriate for the specified program type
*/
public static LoggingContext getLoggingContext(String namespaceId, String appId, String entityId, ProgramType programType);
/**
* Create logging context with run ID for a specific program execution
* @param namespaceId Namespace identifier
* @param appId Application identifier
* @param entityId Program/entity identifier
* @param programType Type of CDAP program
* @param runId Unique identifier for this program run
* @return LoggingContext with run-specific information
*/
public static LoggingContext getLoggingContextWithRunId(String namespaceId, String appId, String entityId, ProgramType programType, String runId);
/**
* Create filter from logging context
* @param loggingContext Context to create filter from
* @return Filter object for log querying based on context
*/
public static Filter createFilter(LoggingContext loggingContext);
/**
* Extract metrics tags from logging context
* @param loggingContext Context to extract tags from
* @return Map of tag names to tag values for metrics tagging
*/
public static Map<String, String> getMetricsTags(LoggingContext loggingContext);
}Usage Examples:
import io.cdap.cdap.logging.context.LoggingContextHelper;
import io.cdap.cdap.logging.context.LoggingContext;
import io.cdap.cdap.common.app.ProgramType;
// Create context for a service
LoggingContext serviceContext = LoggingContextHelper.getLoggingContext(
"myNamespace", // Namespace
"myApplication", // Application
"myService", // Service name
ProgramType.SERVICE // Program type
);
// Create context for a specific workflow run
LoggingContext workflowRunContext = LoggingContextHelper.getLoggingContextWithRunId(
"myNamespace",
"myApplication",
"myWorkflow",
ProgramType.WORKFLOW,
"run-12345" // Specific run ID
);
// Create filter for log queries
Filter logFilter = LoggingContextHelper.createFilter(serviceContext);
// Extract metrics tags for correlation
Map<String, String> metricsTags = LoggingContextHelper.getMetricsTags(serviceContext);Base context class for applications, providing common functionality for all program types.
/**
* Base context for applications
* Provides common logging context functionality for all CDAP programs
*/
public class ApplicationLoggingContext extends AbstractLoggingContext {
/**
* Create application logging context
* @param namespaceId Namespace identifier
* @param applicationId Application identifier
* @param runId Run identifier (optional)
*/
public ApplicationLoggingContext(String namespaceId, String applicationId, String runId);
/**
* Get the log partition for this context
* Used for organizing logs in storage systems
* @return String representing the log partition
*/
public String getLogPartition();
}Logging context for user-defined services with handler-specific tagging.
/**
* Context for user-defined services
* Provides logging context for CDAP user services with handler support
*/
public class UserServiceLoggingContext extends ApplicationLoggingContext {
/** Tag name for user service ID */
public static final String TAG_USER_SERVICE_ID = ".userserviceid";
/** Tag name for handler ID within the service */
public static final String TAG_HANDLER_ID = ".userhandlerid";
/**
* Create user service logging context
* @param namespaceId Namespace identifier
* @param applicationId Application identifier
* @param serviceId User service identifier
* @param runId Run identifier (optional)
*/
public UserServiceLoggingContext(String namespaceId, String applicationId, String serviceId, String runId);
/**
* Get the log partition for this service context
* @return String representing the log partition for this service
*/
public String getLogPartition();
}Logging context for workflow programs with workflow-specific tagging.
/**
* Context for workflow programs
* Provides logging context for CDAP workflow executions
*/
public class WorkflowLoggingContext extends ApplicationLoggingContext {
/** Tag name for workflow ID */
public static final String TAG_WORKFLOW_ID = ".workflowid";
/**
* Create workflow logging context
* @param namespaceId Namespace identifier
* @param applicationId Application identifier
* @param workflowId Workflow identifier
* @param runId Run identifier (optional)
*/
public WorkflowLoggingContext(String namespaceId, String applicationId, String workflowId, String runId);
/**
* Get the log partition for this workflow context
* @return String representing the log partition for this workflow
*/
public String getLogPartition();
}Specialized context for programs running within workflows.
/**
* Context for programs within workflows
* Provides logging context for programs executed as part of a workflow
*/
public class WorkflowProgramLoggingContext extends WorkflowLoggingContext {
/**
* Create workflow program logging context
* @param namespaceId Namespace identifier
* @param applicationId Application identifier
* @param workflowId Workflow identifier
* @param programType Type of program within workflow
* @param programId Program identifier within workflow
* @param runId Run identifier (optional)
*/
public WorkflowProgramLoggingContext(String namespaceId, String applicationId, String workflowId, ProgramType programType, String programId, String runId);
/**
* Get the log partition for this workflow program context
* @return String representing the log partition for this workflow program
*/
public String getLogPartition();
}Logging context for MapReduce programs with job-specific tagging.
/**
* Context for MapReduce programs
* Provides logging context for CDAP MapReduce job executions
*/
public class MapReduceLoggingContext extends ApplicationLoggingContext {
/** Tag name for MapReduce job ID */
public static final String TAG_MAP_REDUCE_JOB_ID = ".mapreducejobid";
/**
* Create MapReduce logging context
* @param namespaceId Namespace identifier
* @param applicationId Application identifier
* @param mapReduceId MapReduce program identifier
* @param runId Run identifier (optional)
*/
public MapReduceLoggingContext(String namespaceId, String applicationId, String mapReduceId, String runId);
/**
* Get the log partition for this MapReduce context
* @return String representing the log partition for this MapReduce job
*/
public String getLogPartition();
}Logging context for Spark programs with job-specific tagging.
/**
* Context for Spark programs
* Provides logging context for CDAP Spark job executions
*/
public class SparkLoggingContext extends ApplicationLoggingContext {
/** Tag name for Spark job ID */
public static final String TAG_SPARK_JOB_ID = ".sparkjobid";
/**
* Create Spark logging context
* @param namespaceId Namespace identifier
* @param applicationId Application identifier
* @param sparkId Spark program identifier
* @param runId Run identifier (optional)
*/
public SparkLoggingContext(String namespaceId, String applicationId, String sparkId, String runId);
/**
* Get the log partition for this Spark context
* @return String representing the log partition for this Spark job
*/
public String getLogPartition();
}Logging context for worker programs with worker-specific tagging.
/**
* Context for worker programs
* Provides logging context for CDAP worker program executions
*/
public class WorkerLoggingContext extends ApplicationLoggingContext {
/** Tag name for worker ID */
public static final String TAG_WORKER_ID = ".workerid";
/**
* Create worker logging context
* @param namespaceId Namespace identifier
* @param applicationId Application identifier
* @param workerId Worker program identifier
* @param runId Run identifier (optional)
*/
public WorkerLoggingContext(String namespaceId, String applicationId, String workerId, String runId);
/**
* Get the log partition for this worker context
* @return String representing the log partition for this worker
*/
public String getLogPartition();
}Context Usage Examples:
import io.cdap.cdap.logging.context.*;
import io.cdap.cdap.common.app.ProgramType;
// Service context with handler
UserServiceLoggingContext serviceContext = new UserServiceLoggingContext(
"production", // Namespace
"data-pipeline", // Application
"http-service", // Service ID
"run-67890" // Run ID
);
// MapReduce context
MapReduceLoggingContext mrContext = new MapReduceLoggingContext(
"analytics",
"batch-processor",
"daily-aggregation",
"run-11111"
);
// Workflow with embedded program
WorkflowLoggingContext workflowContext = new WorkflowLoggingContext(
"etl",
"data-ingestion",
"nightly-workflow",
"run-22222"
);
WorkflowProgramLoggingContext workflowProgramContext = new WorkflowProgramLoggingContext(
"etl",
"data-ingestion",
"nightly-workflow",
ProgramType.MAPREDUCE,
"transform-step",
"run-22222"
);
// Extract partition information for storage
String servicePartition = serviceContext.getLogPartition();
String mrPartition = mrContext.getLogPartition();Each logging context class defines public constants for tag names used in log organization and filtering:
// User Service Context Tags
public static final String TAG_USER_SERVICE_ID = ".userserviceid";
public static final String TAG_HANDLER_ID = ".userhandlerid";
// Workflow Context Tags
public static final String TAG_WORKFLOW_ID = ".workflowid";
// MapReduce Context Tags
public static final String TAG_MAP_REDUCE_JOB_ID = ".mapreducejobid";
// Spark Context Tags
public static final String TAG_SPARK_JOB_ID = ".sparkjobid";
// Worker Context Tags
public static final String TAG_WORKER_ID = ".workerid";These tag constants are used internally for consistent log tagging and can be used by external code for log filtering and querying operations.
Install with Tessl CLI
npx tessl i tessl/maven-io-cdap-cdap--cdap-watchdog