0
# Spark Tags
1

2
Spark Tags provides a comprehensive set of Java and Scala annotations used throughout the Apache Spark ecosystem to classify API stability levels and categorize test suites. This module enables clear communication about API maturity and intended usage patterns while supporting organized test execution and filtering.
3

4
## Package Information
5

6
- **Package Name**: spark-tags_2.13
7
- **Package Type**: maven
8
- **Language**: Java/Scala
9
- **Group ID**: org.apache.spark
10
- **Artifact ID**: spark-tags_2.13
11
- **Installation**: Add to your Maven/SBT dependencies
12

13
## Core Imports
14

15
Java:
16
```java
17
import org.apache.spark.annotation.Experimental;
18
import org.apache.spark.annotation.Stable;
19
import org.apache.spark.annotation.Evolving;
20
import org.apache.spark.annotation.DeveloperApi;
21
import org.apache.spark.annotation.AlphaComponent;
22
import org.apache.spark.tags.SlowHiveTest;
23
import org.apache.spark.tags.DockerTest;
24
```
25

26
Scala:
27
```scala
28
import org.apache.spark.annotation._
29
import org.apache.spark.tags._
30
```
31

32
## Basic Usage
33

34
API Stability Classification:
35
```java
36
@Experimental
37
public class MyExperimentalFeature {
38
    @Stable
39
    public String getVersion() {
40
        return "1.0";
41
    }
42
    
43
    @DeveloperApi
44
    public void internalMethod() {
45
        // Developer-only functionality
46
    }
47
}
48
```
49

50
Test Categorization:
51
```java
52
@SlowHiveTest
53
public class HiveIntegrationTest {
54
    @Test
55
    @DockerTest
56
    public void testWithDocker() {
57
        // Test requiring Docker infrastructure
58
    }
59
}
60
```
61

62
## Architecture
63

64
Spark Tags is organized around two main annotation systems:
65

66
- **API Stability System**: Comprehensive annotation hierarchy for marking API maturity and stability guarantees
67
- **Test Categorization System**: ScalaTest-based annotation framework for organizing and filtering test execution
68
- **Version Documentation**: Scala-specific annotation for documenting feature introduction versions
69
- **Retention Policies**: All annotations use RUNTIME retention for reflection-based tooling support
70

71
## Capabilities
72

73
### API Stability Annotations
74

75
Comprehensive annotation system for classifying Apache Spark API stability levels and communicating compatibility guarantees to users and library developers.
76

77
```java { .api }
78
@Documented
79
@Retention(RetentionPolicy.RUNTIME)
80
@Target({ElementType.TYPE, ElementType.FIELD, ElementType.METHOD, ElementType.PARAMETER,
81
        ElementType.CONSTRUCTOR, ElementType.LOCAL_VARIABLE, ElementType.PACKAGE})
82
public @interface Experimental {}
83

84
@Documented
85
@Retention(RetentionPolicy.RUNTIME)
86
@Target({ElementType.TYPE, ElementType.FIELD, ElementType.METHOD, ElementType.PARAMETER,
87
        ElementType.CONSTRUCTOR, ElementType.LOCAL_VARIABLE, ElementType.PACKAGE})
88
public @interface Stable {}
89

90
@Documented
91
@Retention(RetentionPolicy.RUNTIME)
92
@Target({ElementType.TYPE, ElementType.FIELD, ElementType.METHOD, ElementType.PARAMETER,
93
        ElementType.CONSTRUCTOR, ElementType.LOCAL_VARIABLE, ElementType.PACKAGE})
94
public @interface Evolving {}
95

96
@Documented
97
@Retention(RetentionPolicy.RUNTIME)
98
@Target({ElementType.TYPE, ElementType.FIELD, ElementType.METHOD, ElementType.PARAMETER,
99
        ElementType.CONSTRUCTOR, ElementType.LOCAL_VARIABLE, ElementType.PACKAGE})
100
public @interface DeveloperApi {}
101

102
@Retention(RetentionPolicy.RUNTIME)
103
@Target({ElementType.TYPE, ElementType.FIELD, ElementType.METHOD, ElementType.PARAMETER,
104
        ElementType.CONSTRUCTOR, ElementType.LOCAL_VARIABLE, ElementType.PACKAGE})
105
public @interface AlphaComponent {}
106
```
107

108
[API Stability Annotations](./api-stability.md)
109

110
### Test Categorization Annotations
111

112
ScalaTest-based annotation system for categorizing tests by infrastructure requirements, performance characteristics, and integration complexity.
113

114
```java { .api }
115
@TagAnnotation
116
@Retention(RetentionPolicy.RUNTIME)
117
@Target({ElementType.METHOD, ElementType.TYPE})
118
public @interface SlowHiveTest {}
119

120
@TagAnnotation
121
@Retention(RetentionPolicy.RUNTIME)
122
@Target({ElementType.METHOD, ElementType.TYPE})
123
public @interface DockerTest {}
124

125
@TagAnnotation
126
@Retention(RetentionPolicy.RUNTIME)
127
@Target({ElementType.METHOD, ElementType.TYPE})
128
public @interface ExtendedSQLTest {}
129
```
130

131
[Test Categorization](./test-categorization.md)
132

133
### Version Documentation
134

135
Scala annotation for documenting when API features were introduced, supporting API evolution tracking without requiring explicit JavaDoc.
136

137
```scala { .api }
138
private[spark] class Since(version: String) extends StaticAnnotation
139
```
140

141
The `@Since` annotation takes a version string parameter and can be applied to any Scala definition to document when it was introduced to the Spark API.
142

143
## Types
144

145
### Annotation Targets
146

147
```java { .api }
148
// API Stability annotations target all major Java elements
149
ElementType.TYPE          // Classes, interfaces, enums
150
ElementType.FIELD         // Instance and static fields  
151
ElementType.METHOD        // Methods and constructors
152
ElementType.PARAMETER     // Method and constructor parameters
153
ElementType.CONSTRUCTOR   // Constructor declarations
154
ElementType.LOCAL_VARIABLE // Local variables
155
ElementType.PACKAGE       // Package declarations
156

157
// Test annotations target test classes and methods
158
ElementType.METHOD        // Test methods
159
ElementType.TYPE          // Test classes
160
```
161

162
### Retention Policies
163

164
```java { .api }
165
RetentionPolicy.RUNTIME   // Available at runtime via reflection
166
```
167

168
All annotations use `RUNTIME` retention to support tooling, documentation generation, and build-time analysis.