0
# Flink Architecture Tests Production
1

2
Apache Flink architectural tests for production code validation using the ArchUnit framework. This library provides a comprehensive suite of architectural rules and tests that validate code structure, API annotations, Table API conventions, and connector implementations across Flink's modules, ensuring consistent architectural patterns and enforcing coding standards for large-scale distributed stream processing systems.
3

4
## Package Information
5

6
- **Package Name**: flink-architecture-tests-production
7
- **Package Type**: maven
8
- **Language**: Java
9
- **Installation**: Add to `pom.xml` dependencies:
10
  ```xml
11
  <dependency>
12
    <groupId>org.apache.flink</groupId>
13
    <artifactId>flink-architecture-tests-production</artifactId>
14
    <version>2.1.0</version>
15
    <scope>test</scope>
16
  </dependency>
17
  ```
18

19
## Core Imports
20

21
```java
22
import org.apache.flink.architecture.ProductionCodeArchitectureBase;
23
import com.tngtech.archunit.junit.ArchTest;
24
import com.tngtech.archunit.junit.ArchTests;
25
```
26

27
For specific rule classes:
28

29
```java
30
import org.apache.flink.architecture.rules.ApiAnnotationRules;
31
import org.apache.flink.architecture.rules.TableApiRules;
32
import org.apache.flink.architecture.rules.ConnectorRules;
33
```
34

35
## Basic Usage
36

37
```java
38
import org.apache.flink.architecture.ProductionCodeArchitectureBase;
39
import org.apache.flink.architecture.common.ImportOptions;
40
import com.tngtech.archunit.core.importer.ImportOption;
41
import com.tngtech.archunit.junit.AnalyzeClasses;
42
import com.tngtech.archunit.junit.ArchTest;
43
import com.tngtech.archunit.junit.ArchTests;
44

45
@AnalyzeClasses(
46
    packages = "org.apache.flink",
47
    importOptions = {
48
        ImportOption.DoNotIncludeTests.class,
49
        ImportOptions.ExcludeScalaImportOption.class,
50
        ImportOptions.ExcludeShadedImportOption.class
51
    }
52
)
53
public class MyArchitectureTest {
54
    
55
    @ArchTest
56
    public static final ArchTests PRODUCTION_RULES = 
57
        ArchTests.in(ProductionCodeArchitectureBase.class);
58
}
59
```
60

61
## Architecture
62

63
The library is organized around several key components:
64

65
- **Central Test Base**: `ProductionCodeArchitectureBase` aggregates all production code rules
66
- **Rule Categories**: Separate rule classes for different architectural concerns (API, Table API, Connectors)
67
- **ArchUnit Integration**: Built on ArchUnit framework for architectural testing
68
- **Rule Composition**: Rules can be used individually or as complete suites
69
- **Violation Reporting**: Comprehensive violation reporting with actionable feedback
70

71
## Capabilities
72

73
### Production Code Architecture Base
74

75
Central setup class that aggregates all architectural tests for production code. This is the primary entry point for comprehensive architectural validation.
76

77
```java { .api }
78
/**
79
 * Central setup of architectural tests for the production code.
80
 * Architectural tests should include this class via ArchTests.in(Class) to cover the common part.
81
 */
82
public class ProductionCodeArchitectureBase {
83
    /** Tests for API annotation rules */
84
    @ArchTest
85
    public static final ArchTests API_ANNOTATIONS;
86
    
87
    /** Tests for Table API rules */
88
    @ArchTest
89
    public static final ArchTests TABLE_API;
90
    
91
    /** Tests for connector rules */
92
    @ArchTest
93
    public static final ArchTests CONNECTORS;
94
}
95
```
96

97
**Usage Example:**
98

99
```java
100
@ArchTest
101
public static final ArchTests COMMON_TESTS = 
102
    ArchTests.in(ProductionCodeArchitectureBase.class);
103
```
104

105
### API Annotation Rules
106

107
Validates proper usage of Flink's API visibility annotations (@Public, @PublicEvolving, @Internal, etc.) to ensure consistent API design and prevent accidental exposure of internal APIs.
108

109
```java { .api }
110
/**
111
 * Rules for API visibility annotations.
112
 */
113
public class ApiAnnotationRules {
114
    /** Ensures API classes have at least one visibility annotation */
115
    @ArchTest
116
    public static final ArchRule ANNOTATED_APIS;
117
    
118
    /** Validates @Public methods use only public types */
119
    @ArchTest
120
    public static final ArchRule PUBLIC_API_METHODS_USE_ONLY_PUBLIC_API_TYPES;
121
    
122
    /** Validates @PublicEvolving methods use appropriate types */
123
    @ArchTest
124
    public static final ArchRule PUBLIC_EVOLVING_API_METHODS_USE_ONLY_PUBLIC_EVOLVING_API_TYPES;
125
    
126
    /** Prevents calls to @VisibleForTesting methods from production code */
127
    @ArchTest
128
    public static final ArchRule NO_CALLS_TO_VISIBLE_FOR_TESTING_METHODS;
129
}
130
```
131

132
**Usage Example:**
133

134
```java
135
@ArchTest
136
public static final ArchTests API_RULES = 
137
    ArchTests.in(ApiAnnotationRules.class);
138
```
139

140
### Table API Rules
141

142
Enforces architectural patterns specific to Flink's Table API modules, including configuration option placement, factory implementations, and connector option packaging.
143

144
```java { .api }
145
/**
146
 * Rules for Table API.
147
 */
148
public class TableApiRules {
149
    /** Fully qualified name for ConfigOption type */
150
    public static final String CONFIG_OPTIONS_FQ_NAME = 
151
        "org.apache.flink.configuration.ConfigOption";
152
    
153
    /** Ensures config options are placed in classes ending with "Options" */
154
    @ArchTest
155
    public static final ArchRule CONFIG_OPTIONS_IN_OPTIONS_CLASSES;
156
    
157
    /** Prevents config options in table factory classes */
158
    @ArchTest
159
    public static final ArchRule TABLE_FACTORIES_CONTAIN_NO_CONFIG_OPTIONS;
160
    
161
    /** Validates connector options package structure and annotations */
162
    @ArchTest
163
    public static final ArchRule CONNECTOR_OPTIONS_PACKAGE;
164
    
165
    /** Ensures Table API classes have visibility annotations */
166
    @ArchTest
167
    public static final ArchRule ALL_CLASSES_IN_TABLE_API_SHOULD_HAVE_VISIBILITY_ANNOTATIONS;
168
}
169
```
170

171
**Usage Example:**
172

173
```java
174
@ArchTest
175
public static final ArchTests TABLE_RULES = 
176
    ArchTests.in(TableApiRules.class);
177
```
178

179
### Connector Rules
180

181
Validates that Flink connector implementations follow proper architectural patterns and depend only on public APIs when interacting with core Flink modules.
182

183
```java { .api }
184
/**
185
 * Rules for Flink connectors.
186
 */
187
public class ConnectorRules {
188
    /** 
189
     * Ensures connector classes depend only on public APIs outside connector packages
190
     * Note: This rule is known to fail on Java 11 and Java 17 environments
191
     */
192
    @ArchTest
193
    @ArchTag(value = "org.apache.flink.testutils.junit.FailsOnJava11")
194
    @ArchTag(value = "org.apache.flink.testutils.junit.FailsOnJava17")
195
    public static final ArchRule CONNECTOR_CLASSES_ONLY_DEPEND_ON_PUBLIC_API;
196
}
197
```
198

199
**Usage Example:**
200

201
```java
202
@ArchTest
203
public static final ArchTests CONNECTOR_RULES = 
204
    ArchTests.in(ConnectorRules.class);
205
```
206

207
### Architecture Test Runner
208

209
Complete test class demonstrating proper setup and execution of all architectural tests with appropriate configuration.
210

211
```java { .api }
212
/**
213
 * Architecture tests.
214
 */
215
@AnalyzeClasses(
216
    packages = "org.apache.flink",
217
    importOptions = {
218
        ImportOption.DoNotIncludeTests.class,
219
        ImportOptions.ExcludeScalaImportOption.class,
220
        ImportOptions.ExcludeShadedImportOption.class
221
    }
222
)
223
public class ArchitectureTest {
224
    /** Reference to all common production code tests */
225
    @ArchTest
226
    public static final ArchTests COMMON_TESTS;
227
}
228
```
229

230
## Error Handling
231

232
Architectural rule violations are reported through ArchUnit's standard violation reporting mechanism. Common violation types include:
233

234
- **Missing API Annotations**: Classes in API packages without visibility annotations
235
- **Wrong Dependency Usage**: Connectors depending on internal APIs
236
- **Misplaced Configuration**: Config options in incorrect class locations
237
- **Visibility Violations**: Public methods using non-public types
238

239
**Example Violation Report:**
240
```
241
Architecture Violation [Priority: MEDIUM] - Rule 'Classes in API packages should have at least one API visibility annotation.' was violated (2 times):
242
Class <org.apache.flink.api.common.SomeClass> does not fulfill: are directly annotated with at least one of [@Internal, @Experimental, @PublicEvolving, @Public, @Deprecated]
243
Class <org.apache.flink.api.java.AnotherClass> does not fulfill: are directly annotated with at least one of [@Internal, @Experimental, @PublicEvolving, @Public, @Deprecated]
244
```
245

246
## Implementation Details
247

248
**Key Constants Used in Rules:**
249

250
The architectural rules reference several important package patterns and constants that may appear in violation messages:
251

252
- **Connector Packages**: `"org.apache.flink.connector.."`, `"org.apache.flink.streaming.connectors.."` - Package patterns used to identify connector modules
253
- **Utility Packages**: `"org.apache.flink.util.."` - Utility package patterns allowed for connector dependencies  
254
- **Table API Module Pattern**: `".*/flink-table-(api-(bridge-base|java(|-bridge))|common)/.*"` - Regex pattern for identifying Table API modules
255
- **Config Option Type**: `"org.apache.flink.configuration.ConfigOption"` - Fully qualified name used in Table API rules
256
- **Shaded Package Pattern**: `"..shaded.."` - Pattern for identifying shaded/relocated packages
257

258
## Dependencies
259

260
The library includes these key dependencies:
261

262
- **ArchUnit**: Core architectural testing framework
263
- **ArchUnit JUnit5**: JUnit integration for ArchUnit
264
- **Flink Architecture Base**: Common utilities and predicates
265
- **Flink Annotations**: Flink's API visibility annotations
266
- **JUnit Jupiter**: Test execution framework
267

268
## Platform Requirements
269

270
- **Java Version**: Java 8 or higher
271
- **Build Tool**: Maven (with surefire plugin configuration)
272
- **Memory**: Requires adequate heap space for class loading (configured with `-Xmx` settings)
273
- **Architecture**: Designed for large-scale codebases with thousands of classes
274

275
## Platform Compatibility
276

277
**Known Issues:**
278
- **Java 11/17 Compatibility**: The `CONNECTOR_CLASSES_ONLY_DEPEND_ON_PUBLIC_API` rule in `ConnectorRules` is known to fail on Java 11 and Java 17 environments. This is marked with `@ArchTag` annotations and can be excluded if running on these Java versions.
279
- **Memory Requirements**: Architecture tests analyze thousands of classes, requiring sufficient heap memory. The module is configured to run with `forkCount=1` to consolidate memory usage.
280
- **Scala Exclusion**: Tests automatically exclude Scala classes using `ExcludeScalaImportOption` as ArchUnit has limited Scala support.