dbt Cloud Job Artifact Downloader

Build an Airflow DAG that downloads and archives artifacts from completed dbt Cloud job runs. The DAG should retrieve run results and manifest files from a specified job run and save them to a designated output directory for analysis and auditing purposes.

Requirements

Your implementation must:

1. Create an Airflow DAG named dbt_artifact_downloader that accepts the following configuration parameters:

   • run_id: The dbt Cloud job run ID to download artifacts from
   • output_dir: The directory path where artifacts should be saved

2. Download two specific artifacts from the job run:

   • run_results.json: Contains execution results and test outcomes
   • manifest.json: Contains model metadata and dependencies

3. Save each artifact to the output directory with descriptive filenames that include the run ID (e.g., run_results_12345.json, manifest_12345.json)

4. Handle errors gracefully if artifacts cannot be retrieved

5. Use the dbt Cloud connection ID dbt_cloud_default for authentication

Test Cases

• @test When provided with a valid run ID, the DAG successfully downloads both run_results.json and manifest.json files

• @test The downloaded artifact files are saved with filenames that include the run ID

• @test When provided with an output directory path, both artifacts are saved to that directory

Implementation

@generates

API

"""
DAG for downloading dbt Cloud job run artifacts.

This DAG retrieves run_results.json and manifest.json from a specified
dbt Cloud job run and saves them to a local directory.
"""

# DAG definition with required parameters:
# - run_id: dbt Cloud job run ID
# - output_dir: Local directory for saving artifacts
# - dbt_cloud_conn_id: Connection ID (default: 'dbt_cloud_default')

Dependencies { .dependencies }

apache-airflow-providers-dbt-cloud { .dependency }

Provides integration with dbt Cloud for downloading job artifacts.