Apache Airflow Providers Common Compat

A compatibility provider package that provides backward compatibility utilities and shims for Apache Airflow to maintain compatibility across different Airflow versions. It includes compatibility modules for standard operators, triggers, utilities, asset/dataset handling, lineage tracking, security permissions, and OpenLineage integration to ensure smooth migrations between Airflow versions.

Package Information

• Package Name: apache-airflow-providers-common-compat
• Language: Python
• Installation: pip install apache-airflow-providers-common-compat
• Minimum Airflow Version: 2.10.0+

Core Imports

from airflow.providers.common.compat import __version__

Common compatibility utilities:

from airflow.providers.common.compat.version_compat import (
    AIRFLOW_V_3_0_PLUS,
    AIRFLOW_V_3_1_PLUS,
    BaseOperator
)

Assets/Dataset compatibility:

from airflow.providers.common.compat.assets import Asset, AssetAlias

Basic Usage

from airflow.providers.common.compat.version_compat import AIRFLOW_V_3_0_PLUS, BaseOperator
from airflow.providers.common.compat.assets import Asset
from airflow.providers.common.compat.check import require_provider_version

# Check Airflow version compatibility
if AIRFLOW_V_3_0_PLUS:
    # Use Airflow 3.0+ features
    pass

# Use version-compatible BaseOperator
class MyOperator(BaseOperator):
    def execute(self, context):
        pass

# Create assets with version compatibility
my_asset = Asset("s3://bucket/data.csv")

# Require minimum provider version
@require_provider_version("apache-airflow-providers-openlineage", "1.0.1")
def my_function():
    pass

Architecture

The package provides compatibility across Airflow versions through several key patterns:

• Version Detection: Runtime detection of Airflow version to conditionally import appropriate modules
• Compatibility Wrappers: Unified interfaces that work across different Airflow versions
• Asset/Dataset Bridging: Handles the renaming from "Dataset" to "Asset" in Airflow 3.0
• Provider Version Enforcement: Decorators to ensure minimum provider versions are met
• Graceful Fallbacks: No-op implementations when optional dependencies aren't available

Capabilities

Package Information

Package version and metadata constants.

__version__: str

Version Compatibility

Core utilities for detecting Airflow versions and importing version-appropriate modules, including version constants and compatible BaseOperator imports.

def get_base_airflow_version_tuple() -> tuple[int, int, int]: ...
AIRFLOW_V_3_0_PLUS: bool
AIRFLOW_V_3_1_PLUS: bool
BaseOperator: type

Version Compatibility

Asset Management

Version-compatible asset and dataset handling with support for asset aliases, collections, and authorization details that work across Airflow versions.

class Asset: ...
class AssetAlias: ...
class AssetAll: ...
class AssetAny: ...
def expand_alias_to_assets(...): ...

Asset Management

Provider Verification

Decorators and utilities for enforcing minimum version requirements of provider packages and optional dependencies like OpenLineage.

def require_provider_version(provider_name: str, provider_min_version: str): ...
def require_openlineage_version(provider_min_version: str | None = None, client_min_version: str | None = None): ...

Provider Verification

Lineage Entities

Data lineage entities including files, users, tables, columns, and tags with template field support for dynamic content rendering.

class File: ...
class User: ...
class Table: ...
class Column: ...
class Tag: ...
def default_if_none(arg: bool | None) -> bool: ...

Lineage collection utilities:

def get_hook_lineage_collector(): ...

Lineage Entities

Standard Components

Version-compatible standard Airflow operators, triggers, and utilities including PythonOperator, TimeDeltaTrigger, and virtualenv preparation functions.

class PythonOperator(BaseOperator): ...
class ShortCircuitOperator(BaseOperator): ...
class TimeDeltaTrigger: ...
def get_current_context(): ...

Standard Components

OpenLineage Integration

OpenLineage facets, utilities, and compatibility functions for data lineage tracking, including SQL and Spark integration utilities.

class BaseFacet: ...
class Dataset: ...
class RunFacet: ...
def get_openlineage_facets_with_sql(...): ...
def inject_parent_job_information_into_spark_properties(...): ...

OpenLineage Integration

Security Permissions

Security resource constants for assets, backfills, and DAG versions that maintain compatibility across Airflow security model changes.

RESOURCE_ASSET: str
RESOURCE_ASSET_ALIAS: str
RESOURCE_BACKFILL: str
RESOURCE_DAG_VERSION: str

Security Permissions

Notifier Compatibility

Version-compatible base notifier class for creating custom notification handlers that work across different Airflow versions.

class BaseNotifier: ...

Notifier Compatibility