CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/maven-org-apache-flink--flink-clients

Apache Flink client library providing programmatic APIs and command-line interfaces for submitting, managing, and monitoring Flink jobs.

Pending
Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Pending

The risk profile of this skill

Overview
Eval results
Files

Apache Flink Clients

Apache Flink Clients provides comprehensive client-side functionality for interacting with Flink clusters. This library includes command-line interfaces, programmatic cluster clients, deployment management, application packaging, and artifact handling capabilities for both batch and stream processing workloads.

Package Information

  • Package Name: flink-clients
  • Package Type: maven
  • Language: Java
  • Group ID: org.apache.flink
  • Artifact ID: flink-clients
  • Version: 2.1.0
  • Installation:
    <dependency>
      <groupId>org.apache.flink</groupId>
      <artifactId>flink-clients</artifactId>
      <version>2.1.0</version>
    </dependency>

Core Imports

// CLI Frontend for command-line operations
import org.apache.flink.client.cli.CliFrontend;

// Core cluster client interface
import org.apache.flink.client.program.ClusterClient;

// Program packaging and execution
import org.apache.flink.client.program.PackagedProgram;

// Deployment management
import org.apache.flink.client.deployment.ClusterDescriptor;
import org.apache.flink.client.deployment.ClusterClientFactory;

// Application mode deployment
import org.apache.flink.client.deployment.application.ApplicationRunner;
import org.apache.flink.client.deployment.application.ApplicationConfiguration;

Basic Usage

import org.apache.flink.client.program.ClusterClient;
import org.apache.flink.client.program.PackagedProgram;
import org.apache.flink.client.program.rest.RestClusterClient;
import org.apache.flink.client.program.rest.RestClusterClientConfiguration;
import org.apache.flink.configuration.Configuration;
import org.apache.flink.configuration.RestOptions;

// Create cluster client configuration
Configuration config = new Configuration();
config.setString(RestOptions.ADDRESS, "localhost");
config.setInteger(RestOptions.PORT, 8081);

// Create REST cluster client
RestClusterClientConfiguration clientConfig = 
    RestClusterClientConfiguration.fromConfiguration(config);
try (ClusterClient<?> client = new RestClusterClient<>(clientConfig, "default")) {
    
    // Package and submit a job
    PackagedProgram program = PackagedProgram.newBuilder()
        .setJarFile(new File("/path/to/job.jar"))
        .setArguments(new String[]{"--input", "input.txt"})
        .build();
    
    // Submit job to cluster
    JobID jobId = ClientUtils.executeProgram(
        new DefaultExecutorServiceLoader(), 
        config, 
        program, 
        false, 
        false
    ).getJobID();
    
    System.out.println("Submitted job: " + jobId);
}

Architecture

Apache Flink Clients is built around several key components:

  • CLI Frontend: CliFrontend provides comprehensive command-line interface for job submission, management, and monitoring
  • Cluster Clients: ClusterClient interface abstracts cluster communication with implementations for REST, standalone, and mini clusters
  • Program Packaging: PackagedProgram handles JAR-based job packaging, classpath management, and main class discovery
  • Deployment Framework: Pluggable deployment system supporting multiple cluster types (standalone, YARN, Kubernetes)
  • Application Mode: Dedicated application deployment for long-running applications with ApplicationRunner and related components
  • Artifact Management: Flexible artifact fetching system supporting local files, HTTP, and distributed file systems

Capabilities

Command-Line Interface

Complete CLI frontend for Flink operations including job submission, cancellation, savepoint management, and cluster monitoring.

public class CliFrontend {
    public static void main(String[] args);
    public int parseAndRun(String[] args);
}

Command-Line Interface

Cluster Client Management

Core interface for programmatic cluster interaction, supporting job submission, status monitoring, and cluster lifecycle management.

public interface ClusterClient<T> extends AutoCloseable {
    T getClusterId();
    Configuration getFlinkConfiguration();
    CompletableFuture<Collection<JobStatusMessage>> listJobs() throws Exception;
    CompletableFuture<JobID> submitJob(ExecutionPlan executionPlan);
    CompletableFuture<JobStatus> getJobStatus(JobID jobId);
    String getWebInterfaceURL();
    void shutDownCluster();
}

Cluster Client Management

Program Packaging and Execution

Functionality for packaging Flink programs in JAR files, managing classpaths, and executing user applications.

public class PackagedProgram implements AutoCloseable {
    public static Builder newBuilder();
    public Class<?> getMainClass() throws ProgramInvocationException;
    public String[] getArguments();
    public void invokeInteractiveModeForExecution() throws ProgramInvocationException;
}

Program Packaging

Deployment Management

Cluster deployment abstraction supporting multiple deployment targets with pluggable factory pattern.

public interface ClusterDescriptor<T> extends AutoCloseable {
    ClusterClientProvider<T> retrieve(T clusterId) throws ClusterRetrieveException;
    ClusterClientProvider<T> deploySessionCluster(ClusterSpecification clusterSpecification) 
        throws ClusterDeploymentException;
}

public interface ClusterClientFactory<T> {
    boolean isCompatibleWith(Configuration configuration);
    ClusterDescriptor<T> createClusterDescriptor(Configuration configuration);
}

Deployment Management

Application Mode Deployment

Specialized deployment mode for long-running applications with dedicated cluster resources and lifecycle management.

public interface ApplicationRunner {
    CompletableFuture<Void> run(
        DispatcherGateway dispatcherGateway,
        ScheduledExecutor scheduledExecutor,
        ApplicationConfiguration applicationConfiguration
    );
}

public class ApplicationConfiguration {
    public static Builder fromConfiguration(Configuration configuration);
    public String[] getProgramArguments();
    public String[] getApplicationArgs();
}

Application Mode

Artifact Management

Flexible system for fetching and managing job artifacts from various sources including local files, HTTP endpoints, and distributed file systems.

public interface ArtifactFetcher {
    CompletableFuture<File> fetch(
        URI uri,
        Configuration flinkConf,
        File targetDir,
        @Nullable String filename
    ) throws Exception;
}

public class ArtifactFetchManager {
    public static ArtifactFetchManager fromConfiguration(Configuration configuration);
    public CompletableFuture<File> fetchArtifact(URI uri, File targetDir) throws Exception;
}

Artifact Management

Types

// Core types used across multiple capabilities
public class JobID extends AbstractID {
    public static JobID generate();
}

public enum JobStatus {
    CREATED, RUNNING, FAILING, FAILED, CANCELLING, CANCELED, FINISHED, RESTARTING, SUSPENDED, RECONCILING
}

public class JobStatusMessage {
    public JobID getJobId();
    public String getJobName();
    public JobStatus getJobState();
    public long getStartTime();
}

public class ClusterSpecification {
    public int getMasterMemoryMB();
    public int getTaskManagerMemoryMB();
    public int getNumberTaskManagers();
}

// Exception types
public class ProgramInvocationException extends FlinkException {
    public ProgramInvocationException(String message);
    public ProgramInvocationException(String message, Throwable cause);
}

public class ClusterDeploymentException extends FlinkException {
    public ClusterDeploymentException(String message);
    public ClusterDeploymentException(String message, Throwable cause);
}

public class ClusterRetrieveException extends FlinkException {
    public ClusterRetrieveException(String message);
    public ClusterRetrieveException(String message, Throwable cause);
}
Workspace
tessl
Visibility
Public
Created
Last updated
Describes
mavenpkg:maven/org.apache.flink/flink-clients@2.1.x
Publish Source
CLI
Badge
tessl/maven-org-apache-flink--flink-clients badge