0
# Flink Clients Test Utils
1
2
A specialized test utility module within the Apache Flink ecosystem that provides test classes for validating user classloader functionality in Flink client components. This module creates test JAR files with different configurations to ensure robust classloading behavior across various deployment scenarios.
3
4
## Package Information
5
6
- **Package Name**: flink-clients-test-utils
7
- **Package Type**: maven
8
- **Language**: Java
9
- **GroupId**: org.apache.flink
10
- **ArtifactId**: flink-clients-test-utils
11
- **Version**: 2.1.0
12
- **Installation**: Include as Maven dependency with `<scope>test</scope>`
13
14
```xml
15
<dependency>
16
<groupId>org.apache.flink</groupId>
17
<artifactId>flink-clients-test-utils</artifactId>
18
<version>2.1.0</version>
19
<scope>test</scope>
20
</dependency>
21
```
22
23
## Core Imports
24
25
```java
26
import org.apache.flink.client.testjar.TestUserClassLoaderJob;
27
import org.apache.flink.client.testjar.TestUserClassLoaderAdditionalArtifact;
28
```
29
30
Note: `TestUserClassLoaderJobLib` is package-private and used internally.
31
32
## Basic Usage
33
34
The primary usage is through the generated test JAR artifacts rather than direct class instantiation:
35
36
```java
37
// Example of running the test job JAR
38
// java -cp test-user-classloader-job-jar.jar org.apache.flink.client.testjar.TestUserClassLoaderJob --arg testValue
39
40
// The job creates a simple data stream and executes it
41
public static void main(String[] args) throws Exception {
42
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
43
44
final DataStreamSource<Integer> source =
45
env.fromData(new TestUserClassLoaderJobLib().getValue(), 1, 2, 3, 4);
46
final SingleOutputStreamOperator<Integer> mapper = source.map(element -> 2 * element);
47
mapper.sinkTo(new DiscardingSink<>());
48
49
ParameterTool parameterTool = ParameterTool.fromArgs(args);
50
env.execute(
51
TestUserClassLoaderJob.class.getCanonicalName()
52
+ "-"
53
+ parameterTool.getRequired("arg"));
54
}
55
```
56
57
## Architecture
58
59
The module uses Maven Shade Plugin to create three separate test JAR artifacts:
60
61
1. **test-user-classloader-job-jar**: Contains the main test job for classloader validation
62
2. **test-user-classloader-job-lib-jar**: Contains support library classes for dependency testing
63
3. **test-user-classloader-additional-artifact-jar**: Contains auxiliary classes for additional artifact testing
64
65
Each JAR is designed to test different aspects of Flink's user classloader functionality when jobs are submitted that are not in the system classpath.
66
67
## Capabilities
68
69
### Main Test Job
70
71
The primary test job class that creates and executes a simple Flink streaming job for classloader validation.
72
73
```java { .api }
74
public class TestUserClassLoaderJob {
75
/**
76
* Main entry point for the test job.
77
* Creates a simple streaming pipeline with data transformation and discarding sink.
78
*
79
* @param args Command line arguments, must include "arg" parameter
80
* @throws Exception If job execution fails or "arg" parameter is missing
81
*/
82
public static void main(String[] args) throws Exception;
83
}
84
```
85
86
### Additional Artifact Testing
87
88
A test class for validating classloading from additional user artifacts, providing simple functionality for testing purposes.
89
90
```java { .api }
91
public class TestUserClassLoaderAdditionalArtifact {
92
/**
93
* Returns a constant integer value for testing purposes.
94
* Package-private method for internal testing use.
95
*
96
* @return Always returns 1
97
*/
98
int getNum(); // package-private
99
}
100
```
101
102
### Support Library (Package-Private)
103
104
Internal support class used by the main test job, not intended for direct external usage.
105
106
```java { .api }
107
class TestUserClassLoaderJobLib {
108
/**
109
* Returns a constant integer value used by TestUserClassLoaderJob.
110
*
111
* @return Always returns 0
112
*/
113
int getValue();
114
115
/**
116
* Empty main method for testing purposes.
117
*
118
* @param args Command line arguments (unused)
119
*/
120
public static void main(String[] args);
121
}
122
```
123
124
## Test Artifacts
125
126
The module generates three test JAR artifacts through Maven Shade Plugin configuration:
127
128
### test-user-classloader-job-jar
129
- **Classifier**: job-jar
130
- **Main Class**: org.apache.flink.client.testjar.TestUserClassLoaderJob
131
- **Purpose**: Primary test job for user classloader validation
132
- **Usage**: Submit as a user job to test Flink's ability to load jobs not in system classpath
133
134
**Testing Example:**
135
```bash
136
# Submit job to Flink cluster to test user classloader isolation
137
./bin/flink run \
138
--class org.apache.flink.client.testjar.TestUserClassLoaderJob \
139
test-user-classloader-job-jar.jar \
140
--arg myTestValue
141
```
142
143
### test-user-classloader-job-lib-jar
144
- **Classifier**: job-lib-jar
145
- **Contents**: TestUserClassLoaderJobLib class
146
- **Purpose**: Test dependency resolution within user-submitted JAR files
147
- **Testing Scenario**: Used to verify that dependencies within user JARs are correctly isolated and accessible
148
149
### test-user-classloader-additional-artifact-jar
150
- **Classifier**: additional-artifact-jar
151
- **Contents**: TestUserClassLoaderAdditionalArtifact class
152
- **Purpose**: Test loading of auxiliary classes from additional user artifacts
153
- **Testing Scenario**: Used to validate classloading when additional artifacts are provided alongside the main job JAR
154
155
## Dependencies
156
157
This module has minimal dependencies:
158
159
- **flink-streaming-java** (version 2.1.0, scope: provided) - Core Flink streaming API
160
161
The `provided` scope ensures these dependencies are available at runtime through the Flink cluster environment but not bundled in the test JAR artifacts.
162
163
## Error Handling
164
165
The main test job will throw exceptions in the following scenarios:
166
167
- **Missing "arg" parameter**: `ParameterTool.getRequired("arg")` will throw if the required argument is not provided
168
- **Job execution failure**: Any runtime exceptions during Flink job execution will propagate up
169
- **Environment setup issues**: Problems with StreamExecutionEnvironment creation or configuration
170
171
These exceptions are intentional for testing Flink's error handling and job submission mechanisms.
172
173
## Testing Context and Use Cases
174
175
This module is specifically designed for testing Flink's user classloader functionality in various scenarios:
176
177
### User Classloader Isolation Testing
178
- **Primary Purpose**: Validate that user-submitted jobs are properly isolated from the Flink system classpath
179
- **Test Scenario**: Submit jobs containing these test classes to verify classloader separation
180
- **Expected Behavior**: Jobs should execute successfully without accessing system classpath dependencies
181
182
### Dependency Resolution Testing
183
- **Library JAR Testing**: Use `test-user-classloader-job-lib-jar` to test intra-JAR dependency resolution
184
- **Additional Artifact Testing**: Use `test-user-classloader-additional-artifact-jar` to test multi-JAR scenarios
185
- **Validation**: Ensures user job dependencies are resolved correctly within the user classloader context
186
187
### Integration with Flink Client Tests
188
These utilities are typically used in:
189
- **Client submission tests**: Validating job submission mechanisms handle user classloaders correctly
190
- **Classloader isolation tests**: Ensuring proper separation between user and system code
191
- **End-to-end pipeline tests**: Testing complete job lifecycle with user-provided JARs
192
193
### Typical Testing Workflow
194
1. Build test JARs using Maven Shade Plugin during test compilation
195
2. Submit test jobs programmatically or via Flink CLI to test cluster
196
3. Verify jobs execute correctly without system classpath access
197
4. Validate proper exception handling for missing parameters or execution failures