CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-apache-airflow-providers-hashicorp

Apache Airflow provider package for HashiCorp Vault integration, enabling secret management and authentication within Airflow workflows.

—

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Overview
Eval results
Files

authentication.mddocs/

Authentication

Comprehensive authentication support for HashiCorp Vault with multiple authentication methods. The provider supports all major Vault authentication backends with flexible configuration options for different deployment scenarios.

Capabilities

Supported Authentication Methods

The provider supports the following authentication methods with their corresponding configuration parameters:

SUPPORTED_AUTH_TYPES = [
    "token",        # Token-based authentication
    "approle",      # AppRole authentication
    "github",       # GitHub authentication  
    "gcp",          # Google Cloud Platform authentication
    "kubernetes",   # Kubernetes service account authentication
    "ldap",         # LDAP authentication
    "userpass",     # Username/password authentication
    "aws_iam",      # AWS IAM authentication
    "azure",        # Azure Active Directory authentication
    "radius"        # RADIUS authentication
]

Token Authentication

Direct token-based authentication using Vault tokens.

# Via token parameter
VaultHook(
    vault_conn_id='vault_conn',
    auth_type='token',
    token='your-vault-token'
)

# Via token file
VaultHook(
    vault_conn_id='vault_conn', 
    auth_type='token',
    token_path='/path/to/token/file'
)

Parameters:

  • token (str): Direct token value
  • token_path (str): Path to file containing token

AppRole Authentication

AppRole authentication for applications and services.

VaultHook(
    vault_conn_id='vault_conn',
    auth_type='approle',
    role_id='12345678-1234-1234-1234-123456789012',
    secret_id='abcdef12-3456-7890-abcd-ef1234567890',
    auth_mount_point='approle'  # Optional, default: 'approle'
)

Parameters:

  • role_id (str): AppRole role ID
  • secret_id (str): AppRole secret ID
  • auth_mount_point (str): Mount point for AppRole auth method

Kubernetes Authentication

Authentication using Kubernetes service account tokens.

VaultHook(
    vault_conn_id='vault_conn',
    auth_type='kubernetes',
    kubernetes_role='airflow-role',
    kubernetes_jwt_path='/var/run/secrets/kubernetes.io/serviceaccount/token',
    auth_mount_point='kubernetes'  # Optional, default: 'kubernetes'
)

Parameters:

  • kubernetes_role (str): Kubernetes role bound to service account
  • kubernetes_jwt_path (str): Path to JWT token file
  • auth_mount_point (str): Mount point for Kubernetes auth method

AWS IAM Authentication

Authentication using AWS IAM credentials with optional assume role support.

VaultHook(
    vault_conn_id='vault_conn',
    auth_type='aws_iam',
    role_id='vault-role',
    key_id='AKIAIOSFODNN7EXAMPLE',  # From connection login
    secret_id='wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY',  # From connection password
    region='us-east-1',
    assume_role_kwargs={
        'RoleArn': 'arn:aws:iam::123456789012:role/VaultRole',
        'RoleSessionName': 'airflow-vault-session'
    }
)

Parameters:

  • role_id (str): Vault role for IAM authentication
  • key_id (str): AWS access key ID
  • secret_id (str): AWS secret access key
  • region (str): AWS region for STS calls
  • assume_role_kwargs (dict): Parameters for STS assume role

Google Cloud Authentication

Authentication using Google Cloud Platform service accounts.

# Using key file path
VaultHook(
    vault_conn_id='vault_conn',
    auth_type='gcp',
    gcp_key_path='/path/to/service-account.json',
    gcp_scopes='https://www.googleapis.com/auth/cloud-platform'
)

# Using key dictionary
VaultHook(
    vault_conn_id='vault_conn',
    auth_type='gcp', 
    gcp_keyfile_dict={
        'type': 'service_account',
        'project_id': 'my-project',
        'private_key_id': '...',
        'private_key': '...',
        'client_email': 'service@my-project.iam.gserviceaccount.com',
        'client_id': '...'
    }
)

Parameters:

  • gcp_key_path (str): Path to service account JSON key file
  • gcp_keyfile_dict (dict): Service account key as dictionary
  • gcp_scopes (str): OAuth2 scopes (comma-separated)

Azure Active Directory Authentication

Authentication using Azure AD service principals.

VaultHook(
    vault_conn_id='vault_conn',
    auth_type='azure',
    azure_tenant_id='12345678-1234-1234-1234-123456789012',
    key_id='client-id',  # Client ID
    secret_id='client-secret',  # Client secret
    azure_resource='https://vault.hashicorp.com'
)

Parameters:

  • azure_tenant_id (str): Azure AD tenant ID
  • key_id (str): Azure application client ID
  • secret_id (str): Azure application client secret
  • azure_resource (str): Resource URL for the application

LDAP Authentication

Authentication using LDAP credentials.

VaultHook(
    vault_conn_id='vault_conn',
    auth_type='ldap',
    username='john.doe',  # From connection login
    password='ldap_password',  # From connection password
    auth_mount_point='ldap'  # Optional, default: 'ldap'
)

Parameters:

  • username (str): LDAP username
  • password (str): LDAP password
  • auth_mount_point (str): Mount point for LDAP auth method

Username/Password Authentication

Authentication using Vault's userpass auth method.

VaultHook(
    vault_conn_id='vault_conn',
    auth_type='userpass',
    username='vault_user',  # From connection login
    password='vault_password',  # From connection password
    auth_mount_point='userpass'  # Optional, default: 'userpass'
)

Parameters:

  • username (str): Vault username
  • password (str): Vault password
  • auth_mount_point (str): Mount point for userpass auth method

GitHub Authentication

Authentication using GitHub personal access tokens.

VaultHook(
    vault_conn_id='vault_conn',
    auth_type='github',
    token='github_personal_access_token',  # From connection password
    auth_mount_point='github'  # Optional, default: 'github'
)

Parameters:

  • token (str): GitHub personal access token
  • auth_mount_point (str): Mount point for GitHub auth method

RADIUS Authentication

Authentication using RADIUS protocol.

VaultHook(
    vault_conn_id='vault_conn',
    auth_type='radius',
    username='radius_user',  # From connection login
    radius_secret='shared_secret',
    radius_host='radius.example.com',
    radius_port=1812  # Optional, default: 1812
)

Parameters:

  • username (str): RADIUS username
  • radius_secret (str): RADIUS shared secret
  • radius_host (str): RADIUS server hostname
  • radius_port (int): RADIUS server port

Connection Configuration

Connection Parameters

Authentication parameters can be configured via Airflow connections using the extra field:

{
  "auth_type": "kubernetes",
  "kubernetes_role": "airflow-prod",
  "auth_mount_point": "k8s",
  "kv_engine_version": 2
}

URL Scheme Examples

Vault connection URLs support various authentication schemes:

# Token authentication
vault://user:token@vault.example.com:8200/secret?auth_type=token

# AppRole authentication  
vault://role_id:secret_id@vault.example.com:8200/secret?auth_type=approle

# Kubernetes authentication
vault://service-account:@vault.example.com:8200/secret?auth_type=kubernetes&kubernetes_role=airflow

# AWS IAM authentication
vault://access_key:secret_key@vault.example.com:8200/secret?auth_type=aws_iam&role_id=vault-role

Mount Point Configuration

Each authentication method can use a custom mount point. The following are the default mount points used by the authentication methods:

# Authentication methods use these default mount points when auth_mount_point is not specified
# These defaults are handled internally by the hvac library
DEFAULT_MOUNT_POINTS = {
    'approle': 'approle',
    'aws_iam': 'aws', 
    'azure': 'azure',
    'gcp': 'gcp',
    'github': 'github',
    'kubernetes': 'kubernetes',
    'ldap': 'ldap',
    'radius': 'radius',
    'token': None,  # Token auth does not use a mount point
    'userpass': 'userpass'
}

Override default mount points with the auth_mount_point parameter:

VaultHook(
    auth_type='kubernetes',
    auth_mount_point='k8s-prod',  # Custom mount point
    kubernetes_role='airflow-role'
)

Error Handling

Authentication failures raise appropriate exceptions:

  • VaultError: General Vault authentication failures
  • ConnectionError: Network connectivity issues
  • PermissionError: Insufficient Vault permissions
  • ConfigurationError: Invalid authentication configuration

Handle authentication errors gracefully:

from hvac.exceptions import VaultError

try:
    hook = VaultHook(vault_conn_id='vault_default')
    secret = hook.get_secret('path/to/secret')
except VaultError as e:
    print(f"Vault authentication failed: {e}")
except Exception as e:
    print(f"Unexpected error: {e}")

Install with Tessl CLI

npx tessl i tessl/pypi-apache-airflow-providers-hashicorp

docs

authentication.md

index.md

secrets-backend.md

vault-hook.md

tile.json