dbt Cloud Multi-Step Job Artifact Downloader

Build a utility that downloads artifacts from multi-step dbt Cloud jobs to enable local analysis of test results and model execution.

Capabilities

Retrieve Artifacts from Specific Job Steps

You need to download artifacts from completed dbt Cloud job runs, specifically targeting artifacts from individual steps in multi-step jobs.

• Downloads run_results.json from step 2 of job run ID 12345 for account 67890 and saves it to artifacts/step_2_run_results.json @test
• Downloads manifest.json from step 1 of job run ID 12345 for account 67890 and saves it to artifacts/step_1_manifest.json @test
• Downloads run_results.json from the default step (no step specified) of job run ID 54321 for account 67890 and saves it to artifacts/default_run_results.json @test

Parse Test Results from Artifacts

Extract and count test results from the downloaded run_results.json files.

• Parses run_results.json containing 5 passed tests and 2 failed tests, returning {"passed": 5, "failed": 2} @test
• Parses run_results.json with no test results (only model runs), returning {"passed": 0, "failed": 0} @test

Implementation

@generates

API

def download_step_artifact(
    account_id: int,
    run_id: int,
    artifact_path: str,
    output_file: str,
    step_number: int | None = None,
    dbt_cloud_conn_id: str = "dbt_cloud_default"
) -> None:
    """
    Download an artifact from a dbt Cloud job run, optionally from a specific step.

    Args:
        account_id: The dbt Cloud account ID
        run_id: The job run ID
        artifact_path: Path to the artifact (e.g., 'run_results.json', 'manifest.json')
        output_file: Local file path where the artifact should be saved
        step_number: Optional step number for multi-step jobs (1-indexed)
        dbt_cloud_conn_id: Airflow connection ID for dbt Cloud authentication
    """
    pass

def parse_test_results(run_results_path: str) -> dict[str, int]:
    """
    Parse test results from a run_results.json file.

    Args:
        run_results_path: Path to the run_results.json file

    Returns:
        Dictionary with 'passed' and 'failed' keys containing test counts
    """
    pass

Dependencies { .dependencies }

apache-airflow-providers-dbt-cloud { .dependency }

Provides integration with dbt Cloud API for downloading job run artifacts from specific steps.

@satisfied-by