or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

application-mode.mdartifact-management.mdcli-frontend.mdcluster-client.mddeployment-management.mdindex.mdprogram-packaging.md
tile.json

tessl/maven-org-apache-flink--flink-clients

Apache Flink client library providing programmatic APIs and command-line interfaces for submitting, managing, and monitoring Flink jobs.

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
mavenpkg:maven/org.apache.flink/flink-clients@2.1.x

To install, run

npx @tessl/cli install tessl/maven-org-apache-flink--flink-clients@2.1.0

index.mddocs/

Apache Flink Clients

Apache Flink Clients provides comprehensive client-side functionality for interacting with Flink clusters. This library includes command-line interfaces, programmatic cluster clients, deployment management, application packaging, and artifact handling capabilities for both batch and stream processing workloads.

Package Information

  • Package Name: flink-clients
  • Package Type: maven
  • Language: Java
  • Group ID: org.apache.flink
  • Artifact ID: flink-clients
  • Version: 2.1.0
  • Installation:
    <dependency>
      <groupId>org.apache.flink</groupId>
      <artifactId>flink-clients</artifactId>
      <version>2.1.0</version>
    </dependency>

Core Imports

// CLI Frontend for command-line operations
import org.apache.flink.client.cli.CliFrontend;

// Core cluster client interface
import org.apache.flink.client.program.ClusterClient;

// Program packaging and execution
import org.apache.flink.client.program.PackagedProgram;

// Deployment management
import org.apache.flink.client.deployment.ClusterDescriptor;
import org.apache.flink.client.deployment.ClusterClientFactory;

// Application mode deployment
import org.apache.flink.client.deployment.application.ApplicationRunner;
import org.apache.flink.client.deployment.application.ApplicationConfiguration;

Basic Usage

import org.apache.flink.client.program.ClusterClient;
import org.apache.flink.client.program.PackagedProgram;
import org.apache.flink.client.program.rest.RestClusterClient;
import org.apache.flink.client.program.rest.RestClusterClientConfiguration;
import org.apache.flink.configuration.Configuration;
import org.apache.flink.configuration.RestOptions;

// Create cluster client configuration
Configuration config = new Configuration();
config.setString(RestOptions.ADDRESS, "localhost");
config.setInteger(RestOptions.PORT, 8081);

// Create REST cluster client
RestClusterClientConfiguration clientConfig = 
    RestClusterClientConfiguration.fromConfiguration(config);
try (ClusterClient<?> client = new RestClusterClient<>(clientConfig, "default")) {
    
    // Package and submit a job
    PackagedProgram program = PackagedProgram.newBuilder()
        .setJarFile(new File("/path/to/job.jar"))
        .setArguments(new String[]{"--input", "input.txt"})
        .build();
    
    // Submit job to cluster
    JobID jobId = ClientUtils.executeProgram(
        new DefaultExecutorServiceLoader(), 
        config, 
        program, 
        false, 
        false
    ).getJobID();
    
    System.out.println("Submitted job: " + jobId);
}

Architecture

Apache Flink Clients is built around several key components:

  • CLI Frontend: CliFrontend provides comprehensive command-line interface for job submission, management, and monitoring
  • Cluster Clients: ClusterClient interface abstracts cluster communication with implementations for REST, standalone, and mini clusters
  • Program Packaging: PackagedProgram handles JAR-based job packaging, classpath management, and main class discovery
  • Deployment Framework: Pluggable deployment system supporting multiple cluster types (standalone, YARN, Kubernetes)
  • Application Mode: Dedicated application deployment for long-running applications with ApplicationRunner and related components
  • Artifact Management: Flexible artifact fetching system supporting local files, HTTP, and distributed file systems

Capabilities

Command-Line Interface

Complete CLI frontend for Flink operations including job submission, cancellation, savepoint management, and cluster monitoring.

public class CliFrontend {
    public static void main(String[] args);
    public int parseAndRun(String[] args);
}

Command-Line Interface

Cluster Client Management

Core interface for programmatic cluster interaction, supporting job submission, status monitoring, and cluster lifecycle management.

public interface ClusterClient<T> extends AutoCloseable {
    T getClusterId();
    Configuration getFlinkConfiguration();
    CompletableFuture<Collection<JobStatusMessage>> listJobs() throws Exception;
    CompletableFuture<JobID> submitJob(ExecutionPlan executionPlan);
    CompletableFuture<JobStatus> getJobStatus(JobID jobId);
    String getWebInterfaceURL();
    void shutDownCluster();
}

Cluster Client Management

Program Packaging and Execution

Functionality for packaging Flink programs in JAR files, managing classpaths, and executing user applications.

public class PackagedProgram implements AutoCloseable {
    public static Builder newBuilder();
    public Class<?> getMainClass() throws ProgramInvocationException;
    public String[] getArguments();
    public void invokeInteractiveModeForExecution() throws ProgramInvocationException;
}

Program Packaging

Deployment Management

Cluster deployment abstraction supporting multiple deployment targets with pluggable factory pattern.

public interface ClusterDescriptor<T> extends AutoCloseable {
    ClusterClientProvider<T> retrieve(T clusterId) throws ClusterRetrieveException;
    ClusterClientProvider<T> deploySessionCluster(ClusterSpecification clusterSpecification) 
        throws ClusterDeploymentException;
}

public interface ClusterClientFactory<T> {
    boolean isCompatibleWith(Configuration configuration);
    ClusterDescriptor<T> createClusterDescriptor(Configuration configuration);
}

Deployment Management

Application Mode Deployment

Specialized deployment mode for long-running applications with dedicated cluster resources and lifecycle management.

public interface ApplicationRunner {
    CompletableFuture<Void> run(
        DispatcherGateway dispatcherGateway,
        ScheduledExecutor scheduledExecutor,
        ApplicationConfiguration applicationConfiguration
    );
}

public class ApplicationConfiguration {
    public static Builder fromConfiguration(Configuration configuration);
    public String[] getProgramArguments();
    public String[] getApplicationArgs();
}

Application Mode

Artifact Management

Flexible system for fetching and managing job artifacts from various sources including local files, HTTP endpoints, and distributed file systems.

public interface ArtifactFetcher {
    CompletableFuture<File> fetch(
        URI uri,
        Configuration flinkConf,
        File targetDir,
        @Nullable String filename
    ) throws Exception;
}

public class ArtifactFetchManager {
    public static ArtifactFetchManager fromConfiguration(Configuration configuration);
    public CompletableFuture<File> fetchArtifact(URI uri, File targetDir) throws Exception;
}

Artifact Management

Types

// Core types used across multiple capabilities
public class JobID extends AbstractID {
    public static JobID generate();
}

public enum JobStatus {
    CREATED, RUNNING, FAILING, FAILED, CANCELLING, CANCELED, FINISHED, RESTARTING, SUSPENDED, RECONCILING
}

public class JobStatusMessage {
    public JobID getJobId();
    public String getJobName();
    public JobStatus getJobState();
    public long getStartTime();
}

public class ClusterSpecification {
    public int getMasterMemoryMB();
    public int getTaskManagerMemoryMB();
    public int getNumberTaskManagers();
}

// Exception types
public class ProgramInvocationException extends FlinkException {
    public ProgramInvocationException(String message);
    public ProgramInvocationException(String message, Throwable cause);
}

public class ClusterDeploymentException extends FlinkException {
    public ClusterDeploymentException(String message);
    public ClusterDeploymentException(String message, Throwable cause);
}

public class ClusterRetrieveException extends FlinkException {
    public ClusterRetrieveException(String message);
    public ClusterRetrieveException(String message, Throwable cause);
}