or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

Files

docs

cli-project.mdconfiguration.mdcontext-session.mddata-catalog.mdhooks.mdindex.mdipython-integration.mdpipeline-construction.mdpipeline-execution.md

configuration.mddocs/

0

# Configuration Management

1

2

Kedro's configuration system provides flexible loading and management of project settings, parameters, and environment-specific configurations. It supports multiple file formats and enables parameter injection into pipelines.

3

4

## Capabilities

5

6

### Configuration Loaders

7

8

Base abstractions and implementations for loading configuration from various sources with environment support and parameter resolution.

9

10

```python { .api }

11

class AbstractConfigLoader:

12

"""Abstract base class for configuration loaders."""

13

14

def load_and_merge_dir_config(self, config_path, env=None, **kwargs):

15

"""

16

Load and merge configuration from directory.

17

18

Args:

19

config_path (str): Path to configuration directory

20

env (str, optional): Environment name for env-specific configs

21

**kwargs: Additional loader-specific arguments

22

23

Returns:

24

dict: Merged configuration dictionary

25

"""

26

27

def get(self, *patterns, **kwargs):

28

"""

29

Get configuration matching specified patterns.

30

31

Args:

32

*patterns: Configuration key patterns to match

33

**kwargs: Additional filtering arguments

34

35

Returns:

36

dict: Configuration values matching patterns

37

"""

38

39

class OmegaConfigLoader(AbstractConfigLoader):

40

"""Configuration loader using OmegaConf for YAML/JSON configs."""

41

42

def __init__(self, conf_source, base_env="base", default_run_env="local", **kwargs):

43

"""

44

Initialize OmegaConf-based configuration loader.

45

46

Args:

47

conf_source (str): Path to configuration source directory

48

base_env (str): Base environment name (default: "base")

49

default_run_env (str): Default runtime environment (default: "local")

50

**kwargs: Additional OmegaConf configuration options

51

"""

52

```

53

54

### Configuration Exceptions

55

56

Exception classes for configuration-related errors and validation failures.

57

58

```python { .api }

59

class BadConfigException(Exception):

60

"""Raised when configuration is malformed or invalid."""

61

62

class MissingConfigException(Exception):

63

"""Raised when required configuration is missing."""

64

```

65

66

## Usage Examples

67

68

### Basic Configuration Loading

69

70

```python

71

from kedro.config import OmegaConfigLoader

72

73

# Initialize configuration loader

74

config_loader = OmegaConfigLoader(

75

conf_source="conf",

76

base_env="base",

77

default_run_env="local"

78

)

79

80

# Load configuration for specific environment

81

config = config_loader.load_and_merge_dir_config(

82

config_path="conf",

83

env="production"

84

)

85

86

# Access configuration values

87

database_config = config_loader.get("database.*")

88

logging_config = config_loader.get("logging")

89

```

90

91

### Environment-Specific Configuration

92

93

```python

94

from kedro.config import OmegaConfigLoader

95

96

# Configuration structure:

97

# conf/

98

# base/

99

# catalog.yml

100

# parameters.yml

101

# production/

102

# catalog.yml

103

# parameters.yml

104

105

config_loader = OmegaConfigLoader(conf_source="conf", default_run_env="production")

106

107

# Loads base config merged with production overrides

108

catalog_config = config_loader.get("catalog")

109

parameters = config_loader.get("parameters")

110

```

111

112

### Custom Configuration Loader

113

114

```python

115

from kedro.config import AbstractConfigLoader

116

import yaml

117

from pathlib import Path

118

119

class CustomConfigLoader(AbstractConfigLoader):

120

"""Custom configuration loader with additional features."""

121

122

def load_and_merge_dir_config(self, config_path, env=None, **kwargs):

123

"""Load configuration with custom logic."""

124

base_path = Path(config_path) / "base"

125

env_path = Path(config_path) / env if env else None

126

127

# Load base configuration

128

config = {}

129

for yaml_file in base_path.glob("*.yml"):

130

with open(yaml_file) as f:

131

config.update(yaml.safe_load(f))

132

133

# Merge environment-specific config

134

if env_path and env_path.exists():

135

for yaml_file in env_path.glob("*.yml"):

136

with open(yaml_file) as f:

137

config.update(yaml.safe_load(f))

138

139

return config

140

141

# Usage

142

custom_loader = CustomConfigLoader()

143

config = custom_loader.load_and_merge_dir_config("conf", env="staging")

144

```

145

146

## Types

147

148

```python { .api }

149

from typing import Dict, Any, List, Optional, Union

150

151

ConfigDict = Dict[str, Any]

152

ConfigPath = Union[str, Path]

153

EnvName = Optional[str]

154

ConfigPatterns = List[str]

155

```