Apache Airflow Providers HashiCorp

Apache Airflow provider package for HashiCorp Vault integration, enabling secure secret management and authentication within Airflow workflows. This provider allows Airflow to securely retrieve secrets, connections, and configurations from HashiCorp Vault using various authentication methods.

Package Information

• Package Name: apache-airflow-providers-hashicorp
• Package Type: pypi
• Language: Python
• Installation: pip install apache-airflow-providers-hashicorp

Core Imports

from airflow.providers.hashicorp.hooks.vault import VaultHook
from airflow.providers.hashicorp.secrets.vault import VaultBackend

Basic Usage

from airflow.providers.hashicorp.hooks.vault import VaultHook

# Initialize Vault hook with connection ID
vault_hook = VaultHook(vault_conn_id='vault_default')

# Retrieve a secret from Vault
secret = vault_hook.get_secret('path/to/secret')
print(secret)  # {'key': 'value', 'password': 'secret123'}

# Create or update a secret
new_secret = {'username': 'user', 'password': 'newpass'}
response = vault_hook.create_or_update_secret('path/to/new/secret', new_secret)

# Get secret metadata (KV version 2 only)
metadata = vault_hook.get_secret_metadata('path/to/secret')
print(metadata['versions'])  # Version information

Architecture

The provider package follows Airflow's provider architecture pattern with three main components:

• VaultHook: Primary interface for interacting with HashiCorp Vault's Key-Value secret engine
• VaultBackend: Secrets backend that automatically retrieves Airflow connections, variables, and configurations from Vault
• Connection Types: Defines 'vault' connection type for Airflow UI integration

This architecture enables both programmatic access to Vault secrets within DAGs and automatic secret retrieval for Airflow's core functionality.

Capabilities

Vault Hook Operations

Direct interaction with HashiCorp Vault for secret management operations, including reading, writing, and managing secrets with support for both KV version 1 and 2 engines.

class VaultHook:
    def __init__(self, vault_conn_id: str = None, auth_type: str = "token", **kwargs): ...
    def get_secret(self, secret_path: str, secret_version: int | None = None) -> dict | None: ...
    def create_or_update_secret(self, secret_path: str, secret: dict, method: str | None = None, cas: int | None = None): ...
    def get_secret_metadata(self, secret_path: str) -> dict | None: ...

Vault Hook

Secrets Backend Integration

Automatic retrieval of Airflow connections, variables, and configurations from HashiCorp Vault, enabling seamless integration with Airflow's secrets management system.

class VaultBackend:
    def __init__(self, connections_path: str = "connections", variables_path: str = "variables", **kwargs): ...
    def get_connection(self, conn_id: str): ...
    def get_variable(self, key: str) -> str | None: ...
    def get_config(self, key: str) -> str | None: ...

Secrets Backend

Authentication Methods

Support for multiple HashiCorp Vault authentication methods including token, AppRole, Kubernetes, AWS IAM, Azure AD, GCP, LDAP, and more.

# Supported authentication types
SUPPORTED_AUTH_TYPES = [
    "approle", "github", "gcp", "kubernetes", 
    "ldap", "token", "userpass", "aws_iam", 
    "azure", "radius"
]

Authentication