0
# Configuration
1
2
dbt-core provides a configuration system for managing global flags, project settings, and runtime parameters. This system controls dbt's behavior across CLI and programmatic interfaces.
3
4
## Global Flags
5
6
The global flags system manages runtime configuration that affects dbt's execution behavior.
7
8
### Flag Management Functions
9
10
```python { .api }
11
def get_flags():
12
"""
13
Get the current global flags.
14
15
Returns:
16
Namespace: Current global flags object
17
"""
18
19
def set_flags(flags):
20
"""
21
Set the global flags.
22
23
Args:
24
flags: Namespace or Flags object containing configuration
25
"""
26
27
def set_from_args(args: Namespace, project_flags):
28
"""
29
Set flags from command line arguments and project configuration.
30
31
Args:
32
args: Parsed command line arguments
33
project_flags: Project-specific flag configuration
34
"""
35
36
def get_flag_dict():
37
"""
38
Get flag attributes as a dictionary.
39
40
Returns:
41
dict: Dictionary of flag attributes and values
42
"""
43
```
44
45
### Global Flags Object
46
47
The global flags are stored in a namespace object accessible throughout dbt:
48
49
```python { .api }
50
from dbt.flags import GLOBAL_FLAGS
51
52
# Access current flag values
53
GLOBAL_FLAGS.USE_COLORS # Boolean for colored output
54
GLOBAL_FLAGS.LOG_LEVEL # Current log level
55
GLOBAL_FLAGS.THREADS # Number of execution threads
56
```
57
58
## Project Configuration Constants
59
60
Module: `dbt.constants`
61
62
### File Name Constants
63
64
```python { .api }
65
DBT_PROJECT_FILE_NAME = "dbt_project.yml"
66
"""Standard name for dbt project configuration file."""
67
68
PACKAGES_FILE_NAME = "packages.yml"
69
"""File name for package dependencies."""
70
71
DEPENDENCIES_FILE_NAME = "dependencies.yml"
72
"""Alternative file name for package dependencies."""
73
74
PACKAGE_LOCK_FILE_NAME = "package-lock.yml"
75
"""Lock file for pinned package versions."""
76
77
MANIFEST_FILE_NAME = "manifest.json"
78
"""Compiled manifest artifact file name."""
79
80
SEMANTIC_MANIFEST_FILE_NAME = "semantic_manifest.json"
81
"""Semantic layer manifest file name."""
82
83
PARTIAL_PARSE_FILE_NAME = "partial_parse.msgpack"
84
"""Partial parsing cache file name."""
85
86
CATALOGS_FILE_NAME = "catalogs.yml"
87
"""Catalog configuration file name."""
88
89
RUN_RESULTS_FILE_NAME = "run_results.json"
90
"""Run results artifact file name."""
91
92
CATALOG_FILENAME = "catalog.json"
93
"""Generated catalog artifact file name."""
94
95
SOURCE_RESULT_FILE_NAME = "sources.json"
96
"""Source freshness results file name."""
97
```
98
99
### Size and Limit Constants
100
101
```python { .api }
102
MAXIMUM_SEED_SIZE = 1 * 1024 * 1024
103
"""Maximum size for seed files in bytes (1MB)."""
104
105
MAXIMUM_SEED_SIZE_NAME = "1MB"
106
"""Human-readable maximum seed size."""
107
```
108
109
### Placeholder Constants
110
111
```python { .api }
112
DEFAULT_ENV_PLACEHOLDER = "DBT_DEFAULT_PLACEHOLDER"
113
"""Placeholder for default environment variable values."""
114
115
SECRET_PLACEHOLDER = "$$$DBT_SECRET_START$$${}$$$DBT_SECRET_END$$$"
116
"""Template for masking secrets in logs and output."""
117
```
118
119
### URL Constants
120
121
```python { .api }
122
PIN_PACKAGE_URL = "https://docs.getdbt.com/docs/package-management#section-specifying-package-versions"
123
"""Documentation URL for package version pinning."""
124
```
125
126
### Time Spine Constants
127
128
```python { .api }
129
from dbt_semantic_interfaces.type_enums import TimeGranularity
130
131
LEGACY_TIME_SPINE_MODEL_NAME = "metricflow_time_spine"
132
"""Legacy time spine model name for semantic layer."""
133
134
LEGACY_TIME_SPINE_GRANULARITY = TimeGranularity.DAY
135
"""Default granularity for legacy time spine."""
136
137
MINIMUM_REQUIRED_TIME_SPINE_GRANULARITY = TimeGranularity.DAY
138
"""Minimum required time spine granularity."""
139
```
140
141
## Common Flag Attributes
142
143
The flags system includes many configuration options that control dbt's behavior:
144
145
### Execution Flags
146
147
```python
148
# Threading and performance
149
THREADS: int # Number of execution threads
150
SINGLE_THREADED: bool # Force single-threaded execution
151
PARTIAL_PARSE: bool # Enable partial parsing for performance
152
153
# Execution behavior
154
FULL_REFRESH: bool # Force full refresh of incremental models
155
FAIL_FAST: bool # Stop on first failure
156
STORE_FAILURES: bool # Store test failures in database
157
```
158
159
### Output and Logging Flags
160
161
```python
162
# Output control
163
USE_COLORS: bool # Enable colored terminal output
164
USE_COLORS_FILE: bool # Enable colors in log files
165
QUIET: bool # Suppress non-error output
166
PRINT: bool # Print output to console
167
168
# Logging configuration
169
LOG_LEVEL: str # Log level (debug, info, warn, error)
170
LOG_LEVEL_FILE: str # File log level
171
LOG_PATH: str # Path for log files
172
LOG_FORMAT: str # Log format for console
173
LOG_FORMAT_FILE: str # Log format for files
174
LOG_FILE_MAX_BYTES: int # Maximum log file size
175
```
176
177
### Development and Debugging Flags
178
179
```python
180
# Debugging
181
DEBUG: bool # Enable debug mode
182
MACRO_DEBUGGING: bool # Enable macro debugging
183
184
# Development features
185
USE_EXPERIMENTAL_PARSER: bool # Use experimental parser
186
STATIC_PARSER: bool # Use static parser
187
SHOW_ALL_DEPRECATIONS: bool # Show all deprecation warnings
188
WARN_ERROR: bool # Treat warnings as errors
189
```
190
191
### Selection and Filtering Flags
192
193
```python
194
# Resource selection
195
SELECT: List[str] # Resource selection criteria
196
EXCLUDE: List[str] # Resource exclusion criteria
197
RESOURCE_TYPE: List[str] # Filter by resource type
198
SELECTOR: str # Named selector to use
199
200
# State comparison
201
STATE: str # Path to state directory
202
DEFER: bool # Defer to state for unselected resources
203
FAVOR_STATE: bool # Favor state over current project
204
```
205
206
## Usage Examples
207
208
### Setting Flags Programmatically
209
210
```python
211
from dbt.flags import set_flags
212
from argparse import Namespace
213
214
# Create custom flags
215
custom_flags = Namespace()
216
custom_flags.USE_COLORS = False
217
custom_flags.LOG_LEVEL = 'debug'
218
custom_flags.THREADS = 8
219
220
# Apply flags
221
set_flags(custom_flags)
222
```
223
224
### Accessing Current Configuration
225
226
```python
227
from dbt.flags import get_flags
228
229
flags = get_flags()
230
print(f"Current log level: {flags.LOG_LEVEL}")
231
print(f"Thread count: {flags.THREADS}")
232
print(f"Colors enabled: {flags.USE_COLORS}")
233
```
234
235
### Using Constants
236
237
```python
238
from dbt.constants import (
239
DBT_PROJECT_FILE_NAME,
240
MANIFEST_FILE_NAME,
241
MAXIMUM_SEED_SIZE
242
)
243
import os
244
245
# Check for project file
246
if os.path.exists(DBT_PROJECT_FILE_NAME):
247
print("Found dbt project")
248
249
# Check manifest
250
manifest_path = os.path.join('target', MANIFEST_FILE_NAME)
251
if os.path.exists(manifest_path):
252
print("Manifest available")
253
254
# Validate seed file size
255
def validate_seed_size(file_path):
256
size = os.path.getsize(file_path)
257
if size > MAXIMUM_SEED_SIZE:
258
raise ValueError(f"Seed file too large: {size} bytes")
259
```
260
261
### Environment-Based Configuration
262
263
```python
264
import os
265
from dbt.constants import DEFAULT_ENV_PLACEHOLDER
266
267
def get_env_or_default(key, default=None):
268
"""Get environment variable or return default."""
269
value = os.getenv(key, DEFAULT_ENV_PLACEHOLDER)
270
if value == DEFAULT_ENV_PLACEHOLDER:
271
return default
272
return value
273
274
# Usage
275
threads = int(get_env_or_default('DBT_THREADS', '4'))
276
target = get_env_or_default('DBT_TARGET', 'dev')
277
```