or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

api-stability.mdindex.mdtest-categorization.md
tile.json

tessl/maven-org-apache-spark--spark-tags-2-13

Spark Project Tags - Annotations for API stability classification and test categorization

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
mavenpkg:maven/org.apache.spark/spark-tags_2.13@4.0.x

To install, run

npx @tessl/cli install tessl/maven-org-apache-spark--spark-tags-2-13@4.0.0

index.mddocs/

Spark Tags

Spark Tags provides a comprehensive set of Java and Scala annotations used throughout the Apache Spark ecosystem to classify API stability levels and categorize test suites. This module enables clear communication about API maturity and intended usage patterns while supporting organized test execution and filtering.

Package Information

  • Package Name: spark-tags_2.13
  • Package Type: maven
  • Language: Java/Scala
  • Group ID: org.apache.spark
  • Artifact ID: spark-tags_2.13
  • Installation: Add to your Maven/SBT dependencies

Core Imports

Java:

import org.apache.spark.annotation.Experimental;
import org.apache.spark.annotation.Stable;
import org.apache.spark.annotation.Evolving;
import org.apache.spark.annotation.DeveloperApi;
import org.apache.spark.annotation.AlphaComponent;
import org.apache.spark.tags.SlowHiveTest;
import org.apache.spark.tags.DockerTest;

Scala:

import org.apache.spark.annotation._
import org.apache.spark.tags._

Basic Usage

API Stability Classification:

@Experimental
public class MyExperimentalFeature {
    @Stable
    public String getVersion() {
        return "1.0";
    }
    
    @DeveloperApi
    public void internalMethod() {
        // Developer-only functionality
    }
}

Test Categorization:

@SlowHiveTest
public class HiveIntegrationTest {
    @Test
    @DockerTest
    public void testWithDocker() {
        // Test requiring Docker infrastructure
    }
}

Architecture

Spark Tags is organized around two main annotation systems:

  • API Stability System: Comprehensive annotation hierarchy for marking API maturity and stability guarantees
  • Test Categorization System: ScalaTest-based annotation framework for organizing and filtering test execution
  • Version Documentation: Scala-specific annotation for documenting feature introduction versions
  • Retention Policies: All annotations use RUNTIME retention for reflection-based tooling support

Capabilities

API Stability Annotations

Comprehensive annotation system for classifying Apache Spark API stability levels and communicating compatibility guarantees to users and library developers.

@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE, ElementType.FIELD, ElementType.METHOD, ElementType.PARAMETER,
        ElementType.CONSTRUCTOR, ElementType.LOCAL_VARIABLE, ElementType.PACKAGE})
public @interface Experimental {}

@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE, ElementType.FIELD, ElementType.METHOD, ElementType.PARAMETER,
        ElementType.CONSTRUCTOR, ElementType.LOCAL_VARIABLE, ElementType.PACKAGE})
public @interface Stable {}

@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE, ElementType.FIELD, ElementType.METHOD, ElementType.PARAMETER,
        ElementType.CONSTRUCTOR, ElementType.LOCAL_VARIABLE, ElementType.PACKAGE})
public @interface Evolving {}

@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE, ElementType.FIELD, ElementType.METHOD, ElementType.PARAMETER,
        ElementType.CONSTRUCTOR, ElementType.LOCAL_VARIABLE, ElementType.PACKAGE})
public @interface DeveloperApi {}

@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE, ElementType.FIELD, ElementType.METHOD, ElementType.PARAMETER,
        ElementType.CONSTRUCTOR, ElementType.LOCAL_VARIABLE, ElementType.PACKAGE})
public @interface AlphaComponent {}

API Stability Annotations

Test Categorization Annotations

ScalaTest-based annotation system for categorizing tests by infrastructure requirements, performance characteristics, and integration complexity.

@TagAnnotation
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD, ElementType.TYPE})
public @interface SlowHiveTest {}

@TagAnnotation
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD, ElementType.TYPE})
public @interface DockerTest {}

@TagAnnotation
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD, ElementType.TYPE})
public @interface ExtendedSQLTest {}

Test Categorization

Version Documentation

Scala annotation for documenting when API features were introduced, supporting API evolution tracking without requiring explicit JavaDoc.

private[spark] class Since(version: String) extends StaticAnnotation

The @Since annotation takes a version string parameter and can be applied to any Scala definition to document when it was introduced to the Spark API.

Types

Annotation Targets

// API Stability annotations target all major Java elements
ElementType.TYPE          // Classes, interfaces, enums
ElementType.FIELD         // Instance and static fields  
ElementType.METHOD        // Methods and constructors
ElementType.PARAMETER     // Method and constructor parameters
ElementType.CONSTRUCTOR   // Constructor declarations
ElementType.LOCAL_VARIABLE // Local variables
ElementType.PACKAGE       // Package declarations

// Test annotations target test classes and methods
ElementType.METHOD        // Test methods
ElementType.TYPE          // Test classes

Retention Policies

RetentionPolicy.RUNTIME   // Available at runtime via reflection

All annotations use RUNTIME retention to support tooling, documentation generation, and build-time analysis.