0
# Application Framework
1
2
The Application Framework provides the foundation for building CDAP applications, offering base classes, configuration interfaces, and lifecycle management for complete data processing solutions.
3
4
## Core Application Classes
5
6
### Application Interface
7
8
```java { .api }
9
public interface Application<T extends Config> {
10
void configure(ApplicationConfigurer configurer, ApplicationContext<T> context);
11
}
12
```
13
14
Base interface for all CDAP applications. Applications must implement the `configure(ApplicationConfigurer, ApplicationContext)` method to define their structure, programs, datasets, and services using the provided configurer and accessing context information.
15
16
### AbstractApplication
17
18
```java { .api }
19
public abstract class AbstractApplication<T extends Config> implements Application<T> {
20
public abstract void configure();
21
22
// Application metadata
23
protected final void setName(String name);
24
protected final void setDescription(String description);
25
26
// Program management
27
protected final void addMapReduce(MapReduce mapReduce);
28
protected final void addSpark(Spark spark);
29
protected final void addWorkflow(Workflow workflow);
30
protected final void addService(Service service);
31
protected final void addWorker(Worker worker);
32
33
// Dataset management
34
protected final void addDatasetModule(String moduleName, Class<? extends DatasetModule> moduleClass);
35
protected final void createDataset(String datasetName, String datasetType);
36
protected final void createDataset(String datasetName, String datasetType, DatasetProperties properties);
37
38
// Plugin usage
39
protected final void usePlugin(String pluginType, String pluginName, String pluginId, PluginProperties properties);
40
41
// Scheduling
42
protected final void scheduleWorkflow(Schedules schedule, String workflowName);
43
}
44
```
45
46
Base implementation class for CDAP applications. Provides helper methods for configuring applications with programs, datasets, and scheduling.
47
48
### Single Runnable Application
49
50
```java { .api }
51
public class SingleRunnableApplication<T extends Config> extends AbstractApplication<T> {
52
public SingleRunnableApplication(ProgramLifecycle<? extends RuntimeContext> program);
53
54
@Override
55
public void configure();
56
}
57
```
58
59
Simplified application for single-program use cases.
60
61
## Configuration Interfaces
62
63
### ApplicationConfigurer
64
65
```java { .api }
66
public interface ApplicationConfigurer extends DatasetConfigurer, PluginConfigurer {
67
void setName(String name);
68
void setDescription(String description);
69
70
void addMapReduce(MapReduce mapReduce);
71
void addSpark(Spark spark);
72
void addWorkflow(Workflow workflow);
73
void addService(Service service);
74
void addWorker(Worker worker);
75
76
void scheduleWorkflow(Schedules schedule, String workflowName);
77
}
78
```
79
80
Interface for configuring applications during the configure phase.
81
82
### ProgramConfigurer
83
84
```java { .api }
85
public interface ProgramConfigurer {
86
void setName(String name);
87
void setDescription(String description);
88
void setProperties(Map<String, String> properties);
89
void setResources(Resources resources);
90
}
91
```
92
93
Base interface for configuring individual programs.
94
95
### DatasetConfigurer
96
97
```java { .api }
98
public interface DatasetConfigurer {
99
void addDatasetModule(String moduleName, Class<? extends DatasetModule> moduleClass);
100
void createDataset(String datasetName, String datasetType);
101
void createDataset(String datasetName, String datasetType, DatasetProperties properties);
102
void useDataset(String datasetName);
103
}
104
```
105
106
Interface for dataset configuration within applications and programs.
107
108
## Application Context and Lifecycle
109
110
### ApplicationContext
111
112
```java { .api }
113
public interface ApplicationContext<T extends Config> {
114
T getConfig();
115
ApplicationSpecification getApplicationSpecification();
116
String getNamespace();
117
}
118
```
119
120
Runtime context available to applications during configuration.
121
122
### ApplicationSpecification
123
124
```java { .api }
125
public class ApplicationSpecification {
126
public String getName();
127
public String getDescription();
128
public String getConfiguration();
129
public ArtifactId getArtifactId();
130
131
public Map<String, MapReduceSpecification> getMapReduce();
132
public Map<String, SparkSpecification> getSpark();
133
public Map<String, WorkflowSpecification> getWorkflows();
134
public Map<String, ServiceSpecification> getServices();
135
public Map<String, WorkerSpecification> getWorkers();
136
137
public Map<String, DatasetCreationSpec> getDatasets();
138
public Map<String, Plugin> getPlugins();
139
}
140
```
141
142
Complete specification of an application including all its components and configurations.
143
144
## Core Interfaces
145
146
### ProgramLifecycle
147
148
```java { .api }
149
public interface ProgramLifecycle<T extends RuntimeContext> {
150
void initialize(T context) throws Exception;
151
void destroy();
152
}
153
```
154
155
Lifecycle interface for all program types providing initialization and cleanup hooks.
156
157
### RuntimeContext
158
159
```java { .api }
160
public interface RuntimeContext {
161
String getNamespace();
162
String getApplicationName();
163
String getProgramName();
164
ProgramType getProgramType();
165
String getRunId();
166
167
Map<String, String> getRuntimeArguments();
168
Metrics getMetrics();
169
170
PluginContext getPluginContext();
171
ServiceDiscoverer getServiceDiscoverer();
172
Admin getAdmin();
173
}
174
```
175
176
Base runtime context available to all programs during execution.
177
178
## Configuration Classes
179
180
### Config
181
182
```java { .api }
183
public class Config {
184
// Base configuration class for all configurable components
185
// Can be extended to add custom configuration properties
186
}
187
```
188
189
Base class for application and plugin configurations.
190
191
### Resources
192
193
```java { .api }
194
public class Resources {
195
public Resources(int memoryMB);
196
public Resources(int memoryMB, int virtualCores);
197
198
public int getMemoryMB();
199
public int getVirtualCores();
200
}
201
```
202
203
Resource specification for programs and containers.
204
205
## Usage Examples
206
207
### Basic Application
208
209
```java
210
public class MyApplication extends AbstractApplication<Config> {
211
212
@Override
213
public void configure() {
214
setName("MyDataProcessingApp");
215
setDescription("Processes customer data using MapReduce and Spark");
216
217
// Create datasets
218
createDataset("customerData", KeyValueTable.class);
219
createDataset("processedData", TimePartitionedFileSet.class);
220
221
// Add programs
222
addMapReduce(new CustomerDataProcessor());
223
addSpark(new CustomerAnalytics());
224
addService(new CustomerDataService());
225
226
// Schedule workflow
227
scheduleWorkflow(Schedules.builder("DailyProcessing")
228
.setDescription("Daily customer data processing")
229
.createTimeSchedule("0 2 * * *"), "DataProcessingWorkflow");
230
}
231
}
232
```
233
234
### Application with Custom Configuration
235
236
```java
237
public class CustomConfig extends Config {
238
@Description("Input dataset name")
239
private String inputDataset = "input";
240
241
@Description("Processing batch size")
242
private int batchSize = 1000;
243
244
// Getters and setters
245
public String getInputDataset() { return inputDataset; }
246
public int getBatchSize() { return batchSize; }
247
}
248
249
public class ConfigurableApplication extends AbstractApplication<CustomConfig> {
250
251
@Override
252
public void configure() {
253
CustomConfig config = getConfig();
254
255
setName("ConfigurableApp");
256
createDataset(config.getInputDataset(), KeyValueTable.class);
257
258
// Use configuration in program setup
259
addMapReduce(new ConfigurableProcessor(config.getBatchSize()));
260
}
261
}
262
```
263
264
### Single Program Application
265
266
```java
267
public class SimpleWorkerApp extends SingleRunnableApplication<Config> {
268
269
public SimpleWorkerApp() {
270
super(new SimpleDataWorker());
271
}
272
}
273
```