Sqoop Command Runner

A helper that executes an eval-style Sqoop command using a stored Airflow connection and returns the masked CLI that was executed.

Capabilities

Connection-driven command build

• Builds a Sqoop CLI for the given SQL query using host, port, schema, username, password, and extra JSON fields (job_tracker, namenode, libjars, files, archives, password_file) from the connection identified by conn_id; returns the masked command used for logging. @test
• Includes the connection password for execution but masks it in the returned/logged command representation. @test

Execute and surface failures

• Runs the assembled command through the dependency's Sqoop runner; on non-zero exit, raises the dependency's Sqoop error type with the subprocess stderr, and on success returns the masked command string that was executed. @test

Pass-through options

• Accepts optional extra Sqoop options (e.g., verbose mode or explicit driver flags) and appends them after the connection-derived arguments without overwriting them. @test

Implementation

@generates

API

from typing import Any, Dict, List, Optional

class SqoopCommandRunner:
    def __init__(self, conn_id: str) -> None:
        ...

    def run_eval(self, query: str, extra_options: Optional[Dict[str, Any]] = None, verbose: bool = False) -> str:
        """
        Execute a Sqoop eval query using the configured connection.

        Executes the query through the Sqoop integration and returns the sanitized command string that hides credentials while matching what was sent to the Sqoop subprocess.
        Raises the dependency's Sqoop execution exception on failure.
        """

Dependencies { .dependencies }

apache-airflow-backport-providers-apache-sqoop { .dependency }

Airflow provider supplying the Sqoop hook used to build and execute commands with credential masking. @satisfied-by