CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-dbt-core

A command-line tool for data transformation and analytics engineering workflows.

Pending
Overview
Eval results
Files

configuration.mddocs/

Configuration

dbt-core provides a configuration system for managing global flags, project settings, and runtime parameters. This system controls dbt's behavior across CLI and programmatic interfaces.

Global Flags

The global flags system manages runtime configuration that affects dbt's execution behavior.

Flag Management Functions

def get_flags():
    """
    Get the current global flags.
    
    Returns:
        Namespace: Current global flags object
    """

def set_flags(flags):
    """
    Set the global flags.
    
    Args:
        flags: Namespace or Flags object containing configuration
    """

def set_from_args(args: Namespace, project_flags):
    """
    Set flags from command line arguments and project configuration.
    
    Args:
        args: Parsed command line arguments
        project_flags: Project-specific flag configuration
    """

def get_flag_dict():
    """
    Get flag attributes as a dictionary.
    
    Returns:
        dict: Dictionary of flag attributes and values
    """

Global Flags Object

The global flags are stored in a namespace object accessible throughout dbt:

from dbt.flags import GLOBAL_FLAGS

# Access current flag values
GLOBAL_FLAGS.USE_COLORS  # Boolean for colored output
GLOBAL_FLAGS.LOG_LEVEL   # Current log level
GLOBAL_FLAGS.THREADS     # Number of execution threads

Project Configuration Constants

Module: dbt.constants

File Name Constants

DBT_PROJECT_FILE_NAME = "dbt_project.yml"
"""Standard name for dbt project configuration file."""

PACKAGES_FILE_NAME = "packages.yml"
"""File name for package dependencies."""

DEPENDENCIES_FILE_NAME = "dependencies.yml"
"""Alternative file name for package dependencies."""

PACKAGE_LOCK_FILE_NAME = "package-lock.yml"
"""Lock file for pinned package versions."""

MANIFEST_FILE_NAME = "manifest.json"
"""Compiled manifest artifact file name."""

SEMANTIC_MANIFEST_FILE_NAME = "semantic_manifest.json"
"""Semantic layer manifest file name."""

PARTIAL_PARSE_FILE_NAME = "partial_parse.msgpack"
"""Partial parsing cache file name."""

CATALOGS_FILE_NAME = "catalogs.yml"
"""Catalog configuration file name."""

RUN_RESULTS_FILE_NAME = "run_results.json"
"""Run results artifact file name."""

CATALOG_FILENAME = "catalog.json"
"""Generated catalog artifact file name."""

SOURCE_RESULT_FILE_NAME = "sources.json"
"""Source freshness results file name."""

Size and Limit Constants

MAXIMUM_SEED_SIZE = 1 * 1024 * 1024
"""Maximum size for seed files in bytes (1MB)."""

MAXIMUM_SEED_SIZE_NAME = "1MB"
"""Human-readable maximum seed size."""

Placeholder Constants

DEFAULT_ENV_PLACEHOLDER = "DBT_DEFAULT_PLACEHOLDER"
"""Placeholder for default environment variable values."""

SECRET_PLACEHOLDER = "$$$DBT_SECRET_START$$${}$$$DBT_SECRET_END$$$"
"""Template for masking secrets in logs and output."""

URL Constants

PIN_PACKAGE_URL = "https://docs.getdbt.com/docs/package-management#section-specifying-package-versions"
"""Documentation URL for package version pinning."""

Time Spine Constants

from dbt_semantic_interfaces.type_enums import TimeGranularity

LEGACY_TIME_SPINE_MODEL_NAME = "metricflow_time_spine"
"""Legacy time spine model name for semantic layer."""

LEGACY_TIME_SPINE_GRANULARITY = TimeGranularity.DAY
"""Default granularity for legacy time spine."""

MINIMUM_REQUIRED_TIME_SPINE_GRANULARITY = TimeGranularity.DAY
"""Minimum required time spine granularity."""

Common Flag Attributes

The flags system includes many configuration options that control dbt's behavior:

Execution Flags

# Threading and performance
THREADS: int              # Number of execution threads
SINGLE_THREADED: bool     # Force single-threaded execution
PARTIAL_PARSE: bool       # Enable partial parsing for performance

# Execution behavior  
FULL_REFRESH: bool        # Force full refresh of incremental models
FAIL_FAST: bool          # Stop on first failure
STORE_FAILURES: bool     # Store test failures in database

Output and Logging Flags

# Output control
USE_COLORS: bool         # Enable colored terminal output  
USE_COLORS_FILE: bool    # Enable colors in log files
QUIET: bool              # Suppress non-error output
PRINT: bool              # Print output to console

# Logging configuration
LOG_LEVEL: str           # Log level (debug, info, warn, error)
LOG_LEVEL_FILE: str      # File log level
LOG_PATH: str            # Path for log files
LOG_FORMAT: str          # Log format for console
LOG_FORMAT_FILE: str     # Log format for files
LOG_FILE_MAX_BYTES: int  # Maximum log file size

Development and Debugging Flags

# Debugging
DEBUG: bool              # Enable debug mode
MACRO_DEBUGGING: bool    # Enable macro debugging

# Development features
USE_EXPERIMENTAL_PARSER: bool  # Use experimental parser
STATIC_PARSER: bool           # Use static parser
SHOW_ALL_DEPRECATIONS: bool   # Show all deprecation warnings
WARN_ERROR: bool              # Treat warnings as errors

Selection and Filtering Flags

# Resource selection
SELECT: List[str]        # Resource selection criteria
EXCLUDE: List[str]       # Resource exclusion criteria
RESOURCE_TYPE: List[str] # Filter by resource type
SELECTOR: str            # Named selector to use

# State comparison
STATE: str               # Path to state directory
DEFER: bool              # Defer to state for unselected resources
FAVOR_STATE: bool        # Favor state over current project

Usage Examples

Setting Flags Programmatically

from dbt.flags import set_flags
from argparse import Namespace

# Create custom flags
custom_flags = Namespace()
custom_flags.USE_COLORS = False
custom_flags.LOG_LEVEL = 'debug'
custom_flags.THREADS = 8

# Apply flags
set_flags(custom_flags)

Accessing Current Configuration

from dbt.flags import get_flags

flags = get_flags()
print(f"Current log level: {flags.LOG_LEVEL}")
print(f"Thread count: {flags.THREADS}")
print(f"Colors enabled: {flags.USE_COLORS}")

Using Constants

from dbt.constants import (
    DBT_PROJECT_FILE_NAME,
    MANIFEST_FILE_NAME,
    MAXIMUM_SEED_SIZE
)
import os

# Check for project file
if os.path.exists(DBT_PROJECT_FILE_NAME):
    print("Found dbt project")

# Check manifest
manifest_path = os.path.join('target', MANIFEST_FILE_NAME)
if os.path.exists(manifest_path):
    print("Manifest available")

# Validate seed file size
def validate_seed_size(file_path):
    size = os.path.getsize(file_path)
    if size > MAXIMUM_SEED_SIZE:
        raise ValueError(f"Seed file too large: {size} bytes")

Environment-Based Configuration

import os
from dbt.constants import DEFAULT_ENV_PLACEHOLDER

def get_env_or_default(key, default=None):
    """Get environment variable or return default."""
    value = os.getenv(key, DEFAULT_ENV_PLACEHOLDER)
    if value == DEFAULT_ENV_PLACEHOLDER:
        return default
    return value

# Usage
threads = int(get_env_or_default('DBT_THREADS', '4'))
target = get_env_or_default('DBT_TARGET', 'dev')

Install with Tessl CLI

npx tessl i tessl/pypi-dbt-core

docs

artifacts.md

cli-commands.md

configuration.md

exceptions.md

index.md

programmatic-api.md

version.md

tile.json