0
# Flink Clients Test Utils
1

2
A specialized test utility module within the Apache Flink ecosystem that provides test classes for validating user classloader functionality in Flink client components. This module creates test JAR files with different configurations to ensure robust classloading behavior across various deployment scenarios.
3

4
## Package Information
5

6
- **Package Name**: flink-clients-test-utils
7
- **Package Type**: maven
8
- **Language**: Java
9
- **GroupId**: org.apache.flink
10
- **ArtifactId**: flink-clients-test-utils
11
- **Version**: 2.1.0
12
- **Installation**: Include as Maven dependency with `<scope>test</scope>`
13

14
```xml
15
<dependency>
16
    <groupId>org.apache.flink</groupId>
17
    <artifactId>flink-clients-test-utils</artifactId>
18
    <version>2.1.0</version>
19
    <scope>test</scope>
20
</dependency>
21
```
22

23
## Core Imports
24

25
```java
26
import org.apache.flink.client.testjar.TestUserClassLoaderJob;
27
import org.apache.flink.client.testjar.TestUserClassLoaderAdditionalArtifact;
28
```
29

30
Note: `TestUserClassLoaderJobLib` is package-private and used internally.
31

32
## Basic Usage
33

34
The primary usage is through the generated test JAR artifacts rather than direct class instantiation:
35

36
```java
37
// Example of running the test job JAR
38
// java -cp test-user-classloader-job-jar.jar org.apache.flink.client.testjar.TestUserClassLoaderJob --arg testValue
39

40
// The job creates a simple data stream and executes it
41
public static void main(String[] args) throws Exception {
42
    final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
43
    
44
    final DataStreamSource<Integer> source = 
45
            env.fromData(new TestUserClassLoaderJobLib().getValue(), 1, 2, 3, 4);
46
    final SingleOutputStreamOperator<Integer> mapper = source.map(element -> 2 * element);
47
    mapper.sinkTo(new DiscardingSink<>());
48
    
49
    ParameterTool parameterTool = ParameterTool.fromArgs(args);
50
    env.execute(
51
            TestUserClassLoaderJob.class.getCanonicalName() 
52
                    + "-" 
53
                    + parameterTool.getRequired("arg"));
54
}
55
```
56

57
## Architecture
58

59
The module uses Maven Shade Plugin to create three separate test JAR artifacts:
60

61
1. **test-user-classloader-job-jar**: Contains the main test job for classloader validation
62
2. **test-user-classloader-job-lib-jar**: Contains support library classes for dependency testing
63
3. **test-user-classloader-additional-artifact-jar**: Contains auxiliary classes for additional artifact testing
64

65
Each JAR is designed to test different aspects of Flink's user classloader functionality when jobs are submitted that are not in the system classpath.
66

67
## Capabilities
68

69
### Main Test Job
70

71
The primary test job class that creates and executes a simple Flink streaming job for classloader validation.
72

73
```java { .api }
74
public class TestUserClassLoaderJob {
75
    /**
76
     * Main entry point for the test job.
77
     * Creates a simple streaming pipeline with data transformation and discarding sink.
78
     * 
79
     * @param args Command line arguments, must include "arg" parameter
80
     * @throws Exception If job execution fails or "arg" parameter is missing
81
     */
82
    public static void main(String[] args) throws Exception;
83
}
84
```
85

86
### Additional Artifact Testing
87

88
A test class for validating classloading from additional user artifacts, providing simple functionality for testing purposes.
89

90
```java { .api }
91
public class TestUserClassLoaderAdditionalArtifact {
92
    /**
93
     * Returns a constant integer value for testing purposes.
94
     * Package-private method for internal testing use.
95
     * 
96
     * @return Always returns 1
97
     */
98
    int getNum(); // package-private
99
}
100
```
101

102
### Support Library (Package-Private)
103

104
Internal support class used by the main test job, not intended for direct external usage.
105

106
```java { .api }
107
class TestUserClassLoaderJobLib {
108
    /**
109
     * Returns a constant integer value used by TestUserClassLoaderJob.
110
     * 
111
     * @return Always returns 0
112
     */
113
    int getValue();
114
    
115
    /**
116
     * Empty main method for testing purposes.
117
     * 
118
     * @param args Command line arguments (unused)
119
     */
120
    public static void main(String[] args);
121
}
122
```
123

124
## Test Artifacts
125

126
The module generates three test JAR artifacts through Maven Shade Plugin configuration:
127

128
### test-user-classloader-job-jar
129
- **Classifier**: job-jar
130
- **Main Class**: org.apache.flink.client.testjar.TestUserClassLoaderJob  
131
- **Purpose**: Primary test job for user classloader validation
132
- **Usage**: Submit as a user job to test Flink's ability to load jobs not in system classpath
133

134
**Testing Example:**
135
```bash
136
# Submit job to Flink cluster to test user classloader isolation
137
./bin/flink run \
138
  --class org.apache.flink.client.testjar.TestUserClassLoaderJob \
139
  test-user-classloader-job-jar.jar \
140
  --arg myTestValue
141
```
142

143
### test-user-classloader-job-lib-jar
144
- **Classifier**: job-lib-jar
145
- **Contents**: TestUserClassLoaderJobLib class
146
- **Purpose**: Test dependency resolution within user-submitted JAR files
147
- **Testing Scenario**: Used to verify that dependencies within user JARs are correctly isolated and accessible
148

149
### test-user-classloader-additional-artifact-jar
150
- **Classifier**: additional-artifact-jar
151
- **Contents**: TestUserClassLoaderAdditionalArtifact class
152
- **Purpose**: Test loading of auxiliary classes from additional user artifacts
153
- **Testing Scenario**: Used to validate classloading when additional artifacts are provided alongside the main job JAR
154

155
## Dependencies
156

157
This module has minimal dependencies:
158

159
- **flink-streaming-java** (version 2.1.0, scope: provided) - Core Flink streaming API
160

161
The `provided` scope ensures these dependencies are available at runtime through the Flink cluster environment but not bundled in the test JAR artifacts.
162

163
## Error Handling
164

165
The main test job will throw exceptions in the following scenarios:
166

167
- **Missing "arg" parameter**: `ParameterTool.getRequired("arg")` will throw if the required argument is not provided
168
- **Job execution failure**: Any runtime exceptions during Flink job execution will propagate up
169
- **Environment setup issues**: Problems with StreamExecutionEnvironment creation or configuration
170

171
These exceptions are intentional for testing Flink's error handling and job submission mechanisms.
172

173
## Testing Context and Use Cases
174

175
This module is specifically designed for testing Flink's user classloader functionality in various scenarios:
176

177
### User Classloader Isolation Testing
178
- **Primary Purpose**: Validate that user-submitted jobs are properly isolated from the Flink system classpath
179
- **Test Scenario**: Submit jobs containing these test classes to verify classloader separation
180
- **Expected Behavior**: Jobs should execute successfully without accessing system classpath dependencies
181

182
### Dependency Resolution Testing  
183
- **Library JAR Testing**: Use `test-user-classloader-job-lib-jar` to test intra-JAR dependency resolution
184
- **Additional Artifact Testing**: Use `test-user-classloader-additional-artifact-jar` to test multi-JAR scenarios
185
- **Validation**: Ensures user job dependencies are resolved correctly within the user classloader context
186

187
### Integration with Flink Client Tests
188
These utilities are typically used in:
189
- **Client submission tests**: Validating job submission mechanisms handle user classloaders correctly
190
- **Classloader isolation tests**: Ensuring proper separation between user and system code
191
- **End-to-end pipeline tests**: Testing complete job lifecycle with user-provided JARs
192

193
### Typical Testing Workflow
194
1. Build test JARs using Maven Shade Plugin during test compilation
195
2. Submit test jobs programmatically or via Flink CLI to test cluster
196
3. Verify jobs execute correctly without system classpath access
197
4. Validate proper exception handling for missing parameters or execution failures