or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

caching.mdindex.mdleader-election.mdlocking.mdshared-values.md
tile.json

tessl/maven-org-apache-flink--flink-shaded-curator

A shaded JAR library that bundles Apache Curator dependencies for ZooKeeper coordination capabilities within Apache Flink's distributed architecture.

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
mavenpkg:maven/org.apache.flink/flink-shaded-curator@1.10.x

To install, run

npx @tessl/cli install tessl/maven-org-apache-flink--flink-shaded-curator@1.10.0

index.mddocs/

Flink Shaded Curator

Flink Shaded Curator is a shaded JAR library that provides Apache Curator's distributed coordination capabilities specifically designed for Apache Flink's distributed stream processing framework. It bundles Apache Curator dependencies with relocated classes to avoid classpath conflicts, enabling reliable ZooKeeper-based coordination services within Flink applications.

Package Information

  • Package Name: flink-shaded-curator
  • Package Type: maven
  • Language: Java
  • Group ID: org.apache.flink
  • Artifact ID: flink-shaded-curator
  • Version: 1.10.3
  • Installation: Add to Maven dependencies:
<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-shaded-curator</artifactId>
    <version>1.10.3</version>
</dependency>

Core Imports

import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.curator.framework.recipes.leader.LeaderLatch;
import org.apache.curator.framework.recipes.cache.NodeCache;
import org.apache.curator.framework.recipes.shared.SharedCount;
import org.apache.curator.retry.ExponentialBackoffRetry;

Important: This library relocates certain Guava classes to avoid conflicts. If you need to use Guava functions, import from the shaded namespace:

import org.apache.flink.curator.shaded.com.google.common.base.Function;
import org.apache.flink.curator.shaded.com.google.common.base.Predicate;

Basic Usage

import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.curator.framework.recipes.leader.LeaderLatch;
import org.apache.curator.retry.ExponentialBackoffRetry;

// Create a Curator client (typical Flink usage pattern)
CuratorFramework client = CuratorFrameworkFactory.newClient(
    "localhost:2181", 
    new ExponentialBackoffRetry(1000, 3)
);
client.start();

// Leader election (primary use case in Flink)
LeaderLatch leaderLatch = new LeaderLatch(client, "/flink/leader", "jobmanager-1");
leaderLatch.start();

try {
    // Wait to become leader
    leaderLatch.await();
    System.out.println("I am the leader!");
    
    // Do leader work...
    
} finally {
    leaderLatch.close();
}

client.close();

Architecture

Flink Shaded Curator is built around Apache Curator's distributed coordination patterns:

  • Dependency Isolation: Curator libraries are bundled with selected Guava classes relocated to prevent conflicts with other Flink dependencies
  • ZooKeeper Integration: All coordination patterns rely on ZooKeeper for distributed consensus and state management
  • Flink-Specific Usage: Primarily used for JobManager leader election, checkpoint ID coordination, and high availability services
  • Recipe Patterns: Provides high-level abstractions for distributed coordination, though Flink focuses on leader election and caching capabilities
  • Version Compatibility: Bundles Curator 2.12.0 for compatibility with Flink 1.10.3's distributed architecture

Shading Details

This library is specifically designed for Apache Flink and relocates certain dependencies to avoid classpath conflicts:

  • Curator packages: Remain in original org.apache.curator.* namespace and provide full functionality
  • Selected Guava classes: Only essential classes are included and relocated:
    • com.google.common.base.Function → org.apache.flink.curator.shaded.com.google.common.base.Function
    • com.google.common.base.Predicate → org.apache.flink.curator.shaded.com.google.common.base.Predicate
    • com.google.common.reflect.TypeToken → org.apache.flink.curator.shaded.com.google.common.reflect.TypeToken

Note: This library bundles Apache Curator 2.12.0 and is primarily used within Flink for JobManager leader election, checkpoint coordination, and configuration management.

Capabilities

Leader Election

Leader election capabilities for coordinating which process should act as the primary in a distributed system. This is the primary use case within Flink for JobManager coordination.

public class LeaderLatch implements Closeable {
    public LeaderLatch(CuratorFramework client, String latchPath);
    public LeaderLatch(CuratorFramework client, String latchPath, String id);
    public void start() throws Exception;
    public boolean hasLeadership();
    public void await() throws InterruptedException;
}

public class LeaderSelector implements Closeable {
    public LeaderSelector(CuratorFramework client, String mutexPath, LeaderSelectorListener listener);
    public void start() throws IOException;
    public void requeue() throws InterruptedException;
}

Leader Election

Path Caching

Caching mechanisms for ZooKeeper paths to improve performance and reduce network overhead. Used in Flink for monitoring configuration changes and coordinator state.

public class NodeCache implements Closeable {
    public NodeCache(CuratorFramework client, String path);
    public NodeCache(CuratorFramework client, String path, boolean dataIsCompressed);
    public void start() throws Exception;
    public void start(boolean buildInitial) throws Exception;
    public ChildData getCurrentData();
}

public class PathChildrenCache implements Closeable {
    public PathChildrenCache(CuratorFramework client, String path, boolean cacheData);
    public void start() throws Exception;
    public void start(StartMode mode) throws Exception;
    public List<ChildData> getCurrentData();
}

Path Caching

Shared Counters and Values

Shared data structures for maintaining counters and values across distributed processes. Used in Flink for checkpoint ID coordination and configuration sharing.

public class SharedCount implements Closeable {
    public SharedCount(CuratorFramework client, String path, int seedValue);
    public void start() throws Exception;
    public int getCount();
    public boolean trySetCount(VersionedValue<Integer> previous, int newCount) throws Exception;
}

public class SharedValue implements Closeable {
    public SharedValue(CuratorFramework client, String path, byte[] seedValue);
    public void start() throws Exception;
    public VersionedValue<byte[]> getValue() throws Exception;
    public boolean trySetValue(VersionedValue<byte[]> previous, byte[] newValue) throws Exception;
}

Shared Counters and Values

Distributed Locking

Provides locking mechanisms for coordinating access to shared resources across distributed processes. Available but less commonly used in typical Flink deployments.

public interface InterProcessLock {
    void acquire() throws Exception;
    boolean acquire(long time, TimeUnit unit) throws Exception;
    void release() throws Exception;
}

public class InterProcessMutex implements InterProcessLock {
    public InterProcessMutex(CuratorFramework client, String lockPath);
}

Distributed Locking

Types

// Core framework types
public interface CuratorFramework extends Closeable {
    void start();
    void close();
    CuratorFramework.State getState();
    void blockUntilConnected() throws InterruptedException;
}

// Cache data structures  
public class ChildData {
    public String getPath();
    public Stat getStat();
    public byte[] getData();
}

// Versioned value wrapper
public class VersionedValue<T> {
    public T getValue();
    public int getVersion();
}

// Connection and retry policies
public class ExponentialBackoffRetry implements RetryPolicy {
    public ExponentialBackoffRetry(int baseSleepTimeMs, int maxRetries);
    public ExponentialBackoffRetry(int baseSleepTimeMs, int maxRetries, int maxSleepMs);
}

// Listener interfaces
public interface LeaderLatchListener {
    void isLeader();
    void notLeader();
}

public interface NodeCacheListener {
    void nodeChanged() throws Exception;
}

public interface PathChildrenCacheListener {
    void childEvent(CuratorFramework client, PathChildrenCacheEvent event) throws Exception;
}