0
# Flink End-to-End Tests
1

2
The Flink End-to-End Tests module provides specialized testing utilities for validating Apache Flink's classloader behavior and resolution order settings. This package is designed for end-to-end integration testing scenarios, specifically focusing on verifying that Flink's classloader isolation and resolution mechanisms work correctly across different deployment environments.
3

4
## Package Information
5

6
- **Package Name**: flink-end-to-end-tests_2.11
7
- **Package Type**: maven
8
- **Language**: Java
9
- **Installation**: Maven dependency:
10
  ```xml
11
  <dependency>
12
    <groupId>org.apache.flink</groupId>
13
    <artifactId>flink-end-to-end-tests_2.11</artifactId>
14
    <version>1.4.2</version>
15
  </dependency>
16
  ```
17

18
## Core Imports
19

20
```java
21
// Main test classes
22
import org.apache.flink.streaming.tests.ClassLoaderTestProgram;
23
import org.apache.flink.runtime.taskmanager.TaskManager;
24

25
// Required Flink API imports for streaming execution
26
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
27
import org.apache.flink.api.common.functions.MapFunction;
28
import org.apache.flink.api.java.utils.ParameterTool;
29
import org.apache.flink.core.fs.FileSystem;
30

31
// Standard Java imports for resource handling
32
import java.io.InputStream;
33
import java.net.URL;
34
import java.util.Enumeration;
35
import java.util.Properties;
36
```
37

38
## Basic Usage
39

40
```java
41
// Run as a command-line application with required parameters
42
java -cp ClassLoaderTestProgram.jar org.apache.flink.streaming.tests.ClassLoaderTestProgram \
43
  --resolve-order parent-first \
44
  --output /path/to/output/file
45

46
// Or programmatically execute the main method
47
String[] args = {"--resolve-order", "child-first", "--output", "/tmp/test-results.txt"};
48
ClassLoaderTestProgram.main(args);
49
```
50

51
## Architecture
52

53
The testing framework is built around a clever classloader validation mechanism:
54

55
- **ClassLoaderTestProgram**: Main test program that creates a Flink streaming job to validate classloader behavior
56
- **Fake TaskManager**: A substitute class sharing the same package name as Flink's real TaskManager
57
- **Classloader Resolution Testing**: Uses method availability to verify parent-first vs child-first loading behavior
58
- **Resource Loading Validation**: Tests classpath resource loading order through `.version.properties` files
59

60
The test works by attempting to call a method (`getMessage()`) that exists only in the fake TaskManager class. With parent-first classloading, Flink's real TaskManager is loaded (which lacks this method, causing `NoSuchMethodError`). With child-first classloading, the fake TaskManager is loaded (allowing the method call to succeed).
61

62
## Capabilities
63

64
### Classloader Test Program
65

66
Main test program that validates Flink's classloader resolution order settings through a streaming job execution.
67

68
```java { .api }
69
public class ClassLoaderTestProgram {
70
    /**
71
     * Entry point for the classloader test program.
72
     * Creates and executes a Flink streaming job that validates classloader behavior.
73
     * 
74
     * @param args String array of command line arguments containing:
75
     *             --resolve-order: "parent-first" or "child-first" (required)
76
     *             --output: Output file path for test results (required)
77
     * 
78
     * @throws Exception If execution fails, invalid parameters provided, or streaming job execution fails
79
     * @throws RuntimeException If resolve-order is invalid or unexpected classloader behavior occurs
80
     */
81
    public static void main(String[] args) throws Exception;
82
}
83
```
84

85
**Usage Example:**
86

87
```java
88
// Test parent-first classloader behavior
89
String[] parentFirstArgs = {
90
    "--resolve-order", "parent-first",
91
    "--output", "/tmp/parent-first-results.txt"
92
};
93
ClassLoaderTestProgram.main(parentFirstArgs);
94

95
// Test child-first classloader behavior  
96
String[] childFirstArgs = {
97
    "--resolve-order", "child-first", 
98
    "--output", "/tmp/child-first-results.txt"
99
};
100
ClassLoaderTestProgram.main(childFirstArgs);
101
```
102

103
### Test TaskManager Utility
104

105
Fake TaskManager class used for classloader behavior validation, providing a method that doesn't exist in Flink's real TaskManager.
106

107
```java { .api }
108
public class TaskManager {
109
    /**
110
     * Test method used to validate child-first classloader behavior.
111
     * This method only exists in the test TaskManager, not in Flink's real TaskManager.
112
     * 
113
     * @return String Always returns "Hello, World!" when child-first classloading is active
114
     * @throws NoSuchMethodError When parent-first classloading is active (expected behavior)
115
     */
116
    public static String getMessage();
117
}
118
```
119

120
**Usage Example:**
121

122
```java
123
// This will work only with child-first classloading
124
try {
125
    String message = TaskManager.getMessage();
126
    System.out.println("Child-first loading successful: " + message);
127
} catch (NoSuchMethodError e) {
128
    System.out.println("Parent-first loading detected");
129
}
130
```
131

132
## Test Execution Flow
133

134
1. **Parameter Processing**: Parses command-line arguments for resolve-order and output path
135
2. **Stream Creation**: Creates a simple Flink streaming environment processing a single "Hello" element
136
3. **Classloader Testing**: Within the map function:
137
   - Loads `.version.properties` from classpath
138
   - Attempts to call `TaskManager.getMessage()` method
139
   - Validates behavior based on configured resolve-order
140
4. **Result Generation**: Outputs formatted test results including:
141
   - Resolution behavior indicator ("NoSuchMethodError" or "Hello, World!")
142
   - Git URL from properties file
143
   - Complete classpath resource information
144
5. **File Output**: Writes results to specified output file
145

146
## Error Handling
147

148
The test program handles several error conditions:
149

150
- **Missing Parameters**: Throws exception if required `--resolve-order` or `--output` parameters are missing
151
- **Invalid Resolve Order**: Throws `RuntimeException` for resolve-order values other than "parent-first" or "child-first"
152
- **Unexpected Behavior**: Throws `RuntimeException` if parent-first loading doesn't produce expected `NoSuchMethodError`
153
- **Wrong Message**: Throws `RuntimeException` if child-first loading returns incorrect message from fake TaskManager
154

155
## Output Format
156

157
Test execution produces structured output in the format:
158
```
159
[BEHAVIOR_INDICATOR]:[GIT_URL]:[ORDERED_PROPERTIES]
160
```
161

162
Where:
163
- `BEHAVIOR_INDICATOR`: Either "NoSuchMethodError" (parent-first) or "Hello, World!" (child-first)
164
- `GIT_URL`: Git repository URL from `.version.properties`
165
- `ORDERED_PROPERTIES`: Concatenated git URLs from all classpath `.version.properties` resources
166

167
## Maven Integration
168

169
The package provides specialized Maven configuration for creating executable JARs:
170

171
- **ClassLoaderTestProgram.jar**: Self-contained executable JAR with manifest entry pointing to main class
172
- **Includes**: Compiled test classes and `.version.properties` resource file
173
- **Build Integration**: Automated JAR renaming for easier reference in test scripts
174

175
This module is essential for maintaining the reliability and consistency of Flink applications across different deployment environments and ensuring proper isolation of user code from Flink's internal dependencies.