dbt Cloud Job Inventory

Create an Airflow DAG that discovers and lists all dbt Cloud jobs in a project for inventory tracking. The DAG should retrieve jobs from a dbt Cloud account and project, making their IDs available for downstream tasks.

Requirements

Your DAG should include a task that:

1. Retrieves Job List: Connects to dbt Cloud and retrieves all jobs for a specific account and project. The task should output a list of job IDs.

2. Uses Standard Configuration: The task should use the default connection identifier dbt_cloud_default and accept account and project IDs as parameters.

3. Supports Ordered Results: The task should support ordering the job list by a field name (such as 'name').

Implementation

@generates

API

from airflow import DAG
from datetime import datetime

def create_job_inventory_dag(
    dag_id: str,
    account_id: int,
    project_id: int,
    order_by: str = None
) -> DAG:
    """
    Create an Airflow DAG that inventories dbt Cloud jobs.

    Args:
        dag_id: Unique identifier for the DAG
        account_id: dbt Cloud account ID
        project_id: dbt Cloud project ID to list jobs from
        order_by: Optional field name to order results (e.g., 'name')

    Returns:
        Configured Airflow DAG

    Example:
        dag = create_job_inventory_dag(
            dag_id='dbt_job_inventory',
            account_id=12345,
            project_id=67890,
            order_by='name'
        )
    """
    pass

Test Cases

Basic Job Listing

• Creating a DAG with dag_id='test_inventory', account_id=12345, and project_id=67890 produces a DAG with a task that lists jobs for that project @test

Ordered Job Listing

• Creating a DAG with order_by='name' produces a task that orders the job list by name @test

Task Execution

• Executing the inventory task returns a list of integer job IDs @test

Dependencies { .dependencies }

apache-airflow-providers-dbt-cloud { .dependency }

Provides integration between Apache Airflow and dbt Cloud for job orchestration and discovery.

@satisfied-by