0
# Flink Migration Test Utils
1

2
Flink Migration Test Utils provides comprehensive testing utilities specifically designed for Apache Flink's state migration testing framework. It enables developers to generate and validate state snapshots across different Flink versions, ensuring backward compatibility and smooth migration paths for stateful streaming applications.
3

4
## Package Information
5

6
- **Package Name**: flink-migration-test-utils
7
- **Package Type**: Maven
8
- **Language**: Java
9
- **Group ID**: org.apache.flink
10
- **Artifact ID**: flink-migration-test-utils
11
- **Installation**: Add dependency to your `pom.xml`:
12

13
```xml
14
<dependency>
15
    <groupId>org.apache.flink</groupId>
16
    <artifactId>flink-migration-test-utils</artifactId>
17
    <version>2.1.0</version>
18
    <scope>test</scope>
19
</dependency>
20
```
21

22
## Core Imports
23

24
```java
25
import org.apache.flink.test.util.MigrationTest;
26
import org.apache.flink.test.migration.MigrationTestsSnapshotGenerator;
27
import org.apache.flink.test.migration.PublishedVersionUtils;
28
import org.apache.flink.FlinkVersion;
29
```
30

31
## Basic Usage
32

33
### Simple Migration Test
34

35
```java
36
import org.apache.flink.FlinkVersion;
37
import org.apache.flink.test.util.MigrationTest;
38

39
public class MyStateMigrationTest implements MigrationTest {
40
    
41
    @SnapshotsGenerator
42
    public void generateSnapshots(FlinkVersion version) throws Exception {
43
        // Generate test snapshots for the given Flink version
44
        // This method will be called automatically during snapshot generation
45
        createAndSaveSnapshot(version);
46
    }
47
    
48
    private void createAndSaveSnapshot(FlinkVersion version) {
49
        // Implementation to create state snapshots
50
        // Save snapshots to test resources for migration testing
51
    }
52
}
53
```
54

55
### Generate Snapshots via Command Line
56

57
```bash
58
java org.apache.flink.test.migration.MigrationTestsSnapshotGenerator \
59
    --dir /path/to/project \
60
    --version 2.1 \
61
    --prefixes src/test/java,src/test/scala \
62
    --classes com.example.MyMigrationTest
63
```
64

65
## Capabilities
66

67
### Migration Test Interface
68

69
Core interface that migration tests must implement to participate in automated snapshot generation.
70

71
```java { .api }
72
public interface MigrationTest {
73
    static FlinkVersion getMostRecentlyPublishedVersion();
74
    
75
    @Retention(RetentionPolicy.RUNTIME)
76
    @Target(ElementType.METHOD)
77
    @interface SnapshotsGenerator {}
78
    
79
    @Retention(RetentionPolicy.RUNTIME)
80
    @Target(ElementType.METHOD)
81
    @interface ParameterizedSnapshotsGenerator {
82
        String value();
83
    }
84
}
85
```
86

87
The `MigrationTest` interface provides the foundation for state migration testing:
88

89
- **`getMostRecentlyPublishedVersion()`**: Static method that returns the most recently published Flink version
90
- **`@SnapshotsGenerator`**: Annotation to mark methods that generate snapshots. Marked methods must have signature `void methodName(FlinkVersion version)`
91
- **`@ParameterizedSnapshotsGenerator`**: Annotation for parameterized snapshot generation. Takes the name of a parameter method as its value
92

93
### Snapshot Generation Tool
94

95
Command-line tool for automated snapshot generation across migration test classes.
96

97
```java { .api }
98
public class MigrationTestsSnapshotGenerator {
99
    public static void main(String[] args) throws Throwable;
100
}
101
```
102

103
**Command Line Arguments:**
104

105
- `--help`: Display usage information
106
- `--dir <directory>`: Root directory for scanning (required)
107
- `--version <version>`: Target Flink version for snapshot generation (required, format: `1.17`, `v1.17`, `1_17`, `v1_17`)
108
- `--prefixes <paths>`: Comma-separated search paths (optional, default: `"src/test/java,src/test/scala"`)
109
- `--classes <classes>`: Comma-separated qualified class names to process (optional, overrides prefixes)
110

111
The tool automatically discovers migration test classes that:
112
1. Implement `MigrationTest` interface
113
2. Follow naming pattern `*(Test|ITCase).(java|scala)`
114
3. Have class name matching the file name
115

116
### Version Utilities
117

118
Utility for retrieving published Flink version information.
119

120
```java { .api }
121
public class PublishedVersionUtils {
122
    public static FlinkVersion getMostRecentlyPublishedVersion();
123
}
124
```
125

126
Reads the most recently published Flink version from the bundled resource file. Used internally by the `MigrationTest` interface and can be used directly in test implementations.
127

128
### Snapshot Generation Utilities
129

130
Internal utility class for executing snapshot generation methods. This class is used by `MigrationTestsSnapshotGenerator` to invoke the annotated methods in migration test classes.
131

132
```java { .api }
133
class SnapshotGeneratorUtils {
134
    static void executeGenerate(Class<?> migrationTestClass, FlinkVersion flinkVersion) throws Throwable;
135
}
136
```
137

138
**Note**: This is an internal utility class and not intended for direct use by migration test implementers.
139

140
## Advanced Usage Patterns
141

142
### Parameterized Snapshot Generation
143

144
For tests that need to generate multiple snapshots with different parameters:
145

146
```java
147
public class ParameterizedMigrationTest implements MigrationTest {
148
    
149
    public Collection<TestConfiguration> generateTestConfigurations(FlinkVersion version) {
150
        // Return different test configurations for the given version
151
        return Arrays.asList(
152
            new TestConfiguration("config1", someValue1),
153
            new TestConfiguration("config2", someValue2)
154
        );
155
    }
156
    
157
    @ParameterizedSnapshotsGenerator("generateTestConfigurations")
158
    public void generateSnapshots(TestConfiguration config) throws Exception {
159
        // Generate snapshots using the provided configuration
160
        createSnapshotWithConfig(config);
161
    }
162
}
163
```
164

165
### Maven Integration
166

167
Add the generation profile to enable automated snapshot generation:
168

169
```xml
170
<profile>
171
    <id>generate-migration-test-data</id>
172
    <build>
173
        <plugins>
174
            <plugin>
175
                <artifactId>maven-antrun-plugin</artifactId>
176
                <executions>
177
                    <execution>
178
                        <id>generate-migration-test-data</id>
179
                        <phase>package</phase>
180
                        <goals>
181
                            <goal>run</goal>
182
                        </goals>
183
                        <configuration>
184
                            <target>
185
                                <java classname="org.apache.flink.test.migration.MigrationTestsSnapshotGenerator"
186
                                      fork="true" failonerror="true" dir="${project.basedir}">
187
                                    <classpath refid="maven.test.classpath"/>
188
                                    <arg value="--dir"/>
189
                                    <arg line="${project.basedir}"/>
190
                                    <arg value="--version"/>
191
                                    <arg value="${generate.version}"/>
192
                                </java>
193
                            </target>
194
                        </configuration>
195
                    </execution>
196
                </executions>
197
            </plugin>
198
        </plugins>
199
    </build>
200
</profile>
201
```
202

203
Execute with:
204

205
```bash
206
mvn clean package -Pgenerate-migration-test-data -Dgenerate.version=2.1 -nsu -Dfast -DskipTests
207
```
208

209
## Types
210

211
### FlinkVersion
212

213
```java { .api }
214
// From org.apache.flink.FlinkVersion (external dependency)
215
import java.util.Optional;
216
import java.util.Set;
217

218
public enum FlinkVersion {
219
    v1_3("1.3"), v1_4("1.4"), v1_5("1.5"), v1_6("1.6"), v1_7("1.7"),
220
    v1_8("1.8"), v1_9("1.9"), v1_10("1.10"), v1_11("1.11"), v1_12("1.12"),
221
    v1_13("1.13"), v1_14("1.14"), v1_15("1.15"), v1_16("1.16"), v1_17("1.17"),
222
    v1_18("1.18"), v1_19("1.19"), v1_20("1.20"), v2_0("2.0"), v2_1("2.1");
223
    
224
    public static FlinkVersion valueOf(int majorVersion, int minorVersion);
225
    public static Optional<FlinkVersion> byCode(String code);
226
    public static FlinkVersion current();
227
    public static Set<FlinkVersion> rangeOf(FlinkVersion start, FlinkVersion end);
228
    public boolean isNewerVersionThan(FlinkVersion otherVersion);
229
    public String toString();
230
}
231
```
232

233
### Test Annotations
234

235
```java { .api }
236
@Retention(RetentionPolicy.RUNTIME)
237
@Target(ElementType.METHOD)
238
public @interface SnapshotsGenerator {
239
    // Marks methods for simple snapshot generation
240
    // Method signature: void methodName(FlinkVersion version)
241
}
242

243
@Retention(RetentionPolicy.RUNTIME)
244
@Target(ElementType.METHOD)  
245
public @interface ParameterizedSnapshotsGenerator {
246
    String value(); // Name of method that provides parameters
247
    // Method signature: void methodName(T parameter)
248
    // Parameter method signature: Collection<T> methodName(FlinkVersion version)
249
}
250
```
251

252
## Dependencies
253

254
This library requires the following compile-time dependencies:
255

256
- `org.apache.flink:flink-annotations`
257
- `org.apache.flink:flink-test-utils-junit`
258
- `org.junit.vintage:junit-vintage-engine`
259
- `commons-cli:commons-cli`
260

261
The library is designed to be used as a test-scoped dependency and integrates with Flink's testing framework for state migration validation across version upgrades.
262

263
## Error Handling
264

265
The migration test framework provides comprehensive error handling:
266

267
### Common Exceptions
268

269
- **`FileNotFoundException`**: Thrown when the specified root directory does not exist
270
- **`IllegalArgumentException`**: Thrown when version format cannot be parsed (must match pattern `v?([0-9]+)[._]([0-9]+)`)
271
- **`ClassNotFoundException`**: Logged as warning when specified class names cannot be found on classpath
272
- **`RuntimeException`**: Thrown when:
273
  - Cannot read the most recently published version resource file
274
  - Cannot create migration test instance (missing default constructor or @Parameters method)
275
  - Parameterized argument lists have unsupported types
276

277
### Version Format Requirements
278

279
Version strings must match the regex pattern `v?([0-9]+)[._]([0-9]+)`. Valid examples:
280
- `1.17`
281
- `v1.17` 
282
- `1_17`
283
- `v1_17`
284

285
Invalid examples:
286
- `1.17.0` (patch versions not supported)
287
- `latest` (symbolic names not supported)
288
- `1.17-SNAPSHOT` (qualifiers not supported)