tessl/maven-org-apache-flink--flink-scala-shell-2-11

Interactive Scala shell for Apache Flink that provides a REPL environment for developing and testing Flink applications

—

Pending

Overview

Eval results

Files

Command-Line Interface

Name: tessl/maven-org-apache-flink--flink-scala-shell-2-11
Author: tessl

The Flink Scala Shell provides a comprehensive command-line interface for starting interactive Scala sessions with different Flink cluster configurations.

Imports

import org.apache.flink.api.scala.FlinkShell.Config

Capabilities

Main Entry Point

Entry point that parses command-line arguments and starts the shell.

/**
 * Main entry point for the Flink Scala Shell
 * @param args Command-line arguments specifying execution mode and options
 */
def main(args: Array[String]): Unit

Usage Examples:

# Start with local cluster
start-scala-shell.sh local

# Start with remote cluster
start-scala-shell.sh remote localhost 8081

# Start with YARN cluster
start-scala-shell.sh yarn --jobManagerMemory 1024m --taskManagerMemory 2048m

Shell Startup

Initializes and starts the interactive shell with the provided configuration.

/**
 * Starts the Flink Scala Shell with the given configuration
 * @param config Configuration object containing cluster and execution settings
 */
def startShell(config: Config): Unit

Execution Mode Commands

The shell supports three primary execution modes, each with specific command syntax:

Local Mode

Starts a local Flink mini-cluster for development and testing.

start-scala-shell.sh local [options]

Options:

--addclasspath/-a <path/to/jar> - Add external JARs (colon-separated paths)

Example:

start-scala-shell.sh local --addclasspath /path/to/my-lib.jar:/path/to/other-lib.jar

Remote Mode

Connects to an existing remote Flink cluster.

start-scala-shell.sh remote <host> <port> [options]

Required Arguments:

<host> - Remote JobManager host address
<port> - Remote JobManager port number

Options:

--addclasspath/-a <path/to/jar> - Add external JARs (colon-separated paths)

Example:

start-scala-shell.sh remote flink-cluster.example.com 8081 --addclasspath /path/to/deps.jar

YARN Mode

Connects to or creates a YARN cluster for Flink execution.

start-scala-shell.sh yarn [options]

YARN-Specific Options:

--jobManagerMemory/-jm <memory> - Memory allocation for JobManager container (e.g., "1024m", "2g")
--taskManagerMemory/-tm <memory> - Memory allocation per TaskManager container
--name/-nm <name> - Custom application name on YARN
--queue/-qu <queue> - YARN queue for job submission
--slots/-s <slots> - Number of slots per TaskManager
--addclasspath/-a <path/to/jar> - Add external JARs (colon-separated paths)

Example:

start-scala-shell.sh yarn \
  --jobManagerMemory 1024m \
  --taskManagerMemory 2048m \
  --name "My Flink Shell Session" \
  --queue production \
  --slots 4 \
  --addclasspath /path/to/dependencies.jar

Global Options

Options available across all execution modes:

/**
 * Global configuration options available for all execution modes
 */
case class GlobalOptions(
  configDir: Option[String],    // Configuration directory path
  addclasspath: Option[String], // External JAR dependencies
  help: Boolean                 // Show help information
)

Global Command-Line Options:

--configDir <directory> - Specify Flink configuration directory path
--help/-h - Display usage information and exit

Command-Line Parsing

The shell uses scopt library for robust command-line argument parsing with built-in validation and help generation.

/**
 * Command-line parser configuration with subcommands and options
 */
private val parser: scopt.OptionParser[Config] = new scopt.OptionParser[Config]("start-scala-shell.sh") {
  head("Flink Scala Shell")
  
  cmd("local").action((_, c) => c.copy(executionMode = ExecutionMode.LOCAL))
  cmd("remote").action((_, c) => c.copy(executionMode = ExecutionMode.REMOTE))
  cmd("yarn").action((_, c) => c.copy(executionMode = ExecutionMode.YARN))
}

Error Handling

The command-line interface provides comprehensive error handling for common configuration issues:

Missing required arguments: Clear error messages for missing host/port in remote mode
Invalid execution mode: Guidance on correct usage when mode is unspecified
Configuration errors: Detailed error information for cluster connection failures
Help display: Automatic help text generation with usage examples

Example Error Messages:

Error: <host> or <port> is not specified!
Error: please specify execution mode: [local | remote <host> <port> | yarn]
Could not parse program arguments

Configuration Directory

The shell automatically locates Flink configuration using:

--configDir command-line option (highest priority)
FLINK_CONF_DIR environment variable
Default Flink configuration directory resolution

/**
 * Resolves configuration directory from command-line options or environment
 * @param config Configuration object with optional configDir
 * @return Path to configuration directory
 */
private def getConfigDir(config: Config): String

/**
 * Loads global Flink configuration from resolved directory
 * @param config Configuration object
 * @return Loaded Configuration instance
 */
private def getGlobalConfig(config: Config): Configuration

Install with Tessl CLI

npx tessl i tessl/maven-org-apache-flink--flink-scala-shell-2-11

docs

command-line-interface.md

tessl/maven-org-apache-flink--flink-scala-shell-2-11