Formatted Sqoop Export

Build a small utility that triggers an export from an HDFS/Hive directory into a relational table while exercising the dependency's formatting controls for CSV/text output. Rely on the package's export facility to apply delimiters, enclosure/escape rules, and null sentinels rather than invoking raw shell commands directly.

Capabilities

Field and line delimiters

• It uses | as the field separator and \n as the line separator when exporting a sample dataset, and the resulting command reflects those separators. @test

Enclosure and escaping

• Values containing delimiter characters are enclosed with " and embedded quotes are escaped with \\ in the export configuration. @test

Null sentinels

• Null string columns are written as NULL and null numeric columns as \\N, using the export facility's null markers. @test

Override defaults per run

• Callers can override delimiters, enclosure, escape, and null markers for a single run, and the export applies the supplied values without affecting other runs. @test

Implementation

@generates

API

from dataclasses import dataclass
from typing import Optional

@dataclass
class ExportFormatting:
    field_delimiter: str = "|"
    line_delimiter: str = "\n"
    enclosure: Optional[str] = '"'
    escape: str = "\\"
    string_null: str = "NULL"
    non_string_null: str = "\\N"

@dataclass
class ExportRequest:
    connection_id: str
    source_dir: str
    target_table: str
    formatting: ExportFormatting

def run_formatted_export(request: ExportRequest) -> str:
    """
    Executes an export using the dependency's export capability with the provided
    formatting controls. Returns the full command or identifier used for the run
    so tests can assert the applied options.
    """

Dependencies { .dependencies }

apache-airflow-backport-providers-apache-sqoop { .dependency }

Provides the export capability for moving files from HDFS/Hive into a database with customizable delimiters, enclosure/escape rules, and null sentinels.