CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-pystow

Easily pick a place to store data for your Python code with standardized directory management, caching, and data format support.

Pending
Overview
Eval results
Files

configuration.mddocs/

Configuration Management

PyStow provides a flexible configuration system that supports environment variables and INI files for storing API keys, URLs, connection strings, and other application settings. The configuration system follows a hierarchical lookup strategy.

Configuration Functions

Reading Configuration

def get_config(module: str, key: str, *, passthrough: X | None = None, default: X | None = None, dtype: type[X] | None = None, raise_on_missing: bool = False) -> Any:
    """Get a configuration value.
    
    Args:
        module: Name of the module (e.g., pybel) to get configuration for
        key: Name of the key (e.g., connection)
        passthrough: If this is not none, will get returned
        default: If the environment and configuration files don't contain anything,
            this is returned.
        dtype: The datatype to parse out. Can either be int, float,
            bool, or str. If none, defaults to str.
        raise_on_missing: If true, will raise a value error if no data is found and
            no default is given
    
    Returns:
        The config value or the default.
    
    Raises:
        ConfigError: If raise_on_missing conditions are met
    """

Writing Configuration

def write_config(module: str, key: str, value: str) -> None:
    """Write a configuration value.
    
    Args:
        module: The name of the app (e.g., indra)
        key: The key of the configuration in the app
        value: The value of the configuration in the app
    """

Configuration Exceptions

class ConfigError(ValueError):
    """Raised when configuration can not be looked up."""
    
    def __init__(self, module: str, key: str):
        """Initialize the configuration error.
        
        Args:
            module: Name of the module, e.g., bioportal
            key: Name of the key inside the module, e.g., api_key
        """

Configuration Hierarchy

PyStow uses a multi-level configuration lookup system:

  1. Passthrough values (if provided directly)
  2. Environment variables (MODULE_KEY format, uppercase)
  3. INI configuration files (multiple locations searched)
  4. Default values (if provided)

Environment Variable Format

Environment variables follow the pattern: {MODULE}_{KEY} (uppercase)

Examples:

  • Module: myapp, Key: api_key → Environment variable: MYAPP_API_KEY
  • Module: database, Key: host → Environment variable: DATABASE_HOST

INI File Locations

PyStow searches for configuration files in the following order:

  1. ~/.config/config.cfg
  2. ~/.config/config.ini
  3. ~/.config/pystow.cfg
  4. ~/.config/pystow.ini
  5. ~/.config/{module}.cfg
  6. ~/.config/{module}.ini
  7. ~/.config/{module}/{module}.cfg
  8. ~/.config/{module}/{module}.ini
  9. ~/.config/{module}/conf.ini
  10. ~/.config/{module}/config.ini
  11. ~/.config/{module}/conf.cfg
  12. ~/.config/{module}/config.cfg

INI File Format

Configuration files use standard INI format:

[myapp]
api_key = your_api_key_here
base_url = https://api.example.com
timeout = 30
debug = true

[database]
host = localhost
port = 5432
name = myapp_db

Usage Examples

Basic Configuration Usage

import pystow

# Get API key with default
api_key = pystow.get_config(
    "myapp", "api_key",
    default="default_key"
)

# Get configuration with type conversion
timeout = pystow.get_config(
    "myapp", "timeout",
    dtype=int,
    default=30
)

# Get boolean configuration
debug_mode = pystow.get_config(
    "myapp", "debug",
    dtype=bool,
    default=False
)

Setting Configuration

import pystow

# Write configuration values
pystow.write_config("myapp", "api_key", "your_secret_key")
pystow.write_config("myapp", "base_url", "https://api.example.com")
pystow.write_config("myapp", "timeout", "60")

Error Handling

import pystow
from pystow import ConfigError

try:
    # Require configuration value (no default)
    api_key = pystow.get_config(
        "myapp", "api_key",
        raise_on_missing=True
    )
except ConfigError as e:
    print(f"Configuration missing: {e}")
    # ConfigError provides helpful setup instructions

Type Conversion

import pystow

# String values (default)
username = pystow.get_config("database", "username")

# Integer values
port = pystow.get_config("database", "port", dtype=int, default=5432)

# Float values  
timeout = pystow.get_config("api", "timeout", dtype=float, default=30.0)

# Boolean values
ssl_enabled = pystow.get_config("database", "ssl", dtype=bool, default=True)

Environment Variable Configuration

# Set environment variables
export MYAPP_API_KEY="secret_key_123"
export MYAPP_BASE_URL="https://api.example.com"
export MYAPP_TIMEOUT="60"
export MYAPP_DEBUG="true"
import pystow

# These will read from environment variables
api_key = pystow.get_config("myapp", "api_key")        # "secret_key_123"
base_url = pystow.get_config("myapp", "base_url")      # "https://api.example.com"
timeout = pystow.get_config("myapp", "timeout", dtype=int)  # 60
debug = pystow.get_config("myapp", "debug", dtype=bool)     # True

Configuration File Setup

Create ~/.config/myapp.ini:

[myapp]
api_key = your_api_key_here
base_url = https://api.example.com/v1
timeout = 30
debug = false
max_retries = 3

[database]
host = db.example.com
port = 5432
name = production_db
ssl = true
import pystow

# Read from configuration file
api_key = pystow.get_config("myapp", "api_key")
db_host = pystow.get_config("database", "host")
db_port = pystow.get_config("database", "port", dtype=int)
ssl_enabled = pystow.get_config("database", "ssl", dtype=bool)

Advanced Configuration Patterns

import pystow

class DatabaseConfig:
    """Database configuration manager"""
    
    def __init__(self, module_name="database"):
        self.module = module_name
    
    @property
    def host(self):
        return pystow.get_config(self.module, "host", default="localhost")
    
    @property
    def port(self):
        return pystow.get_config(self.module, "port", dtype=int, default=5432)
    
    @property
    def name(self):
        return pystow.get_config(self.module, "name", raise_on_missing=True)
    
    @property
    def connection_string(self):
        return f"postgresql://{self.host}:{self.port}/{self.name}"

# Usage
db_config = DatabaseConfig()
conn_str = db_config.connection_string

Multi-Environment Configuration

import pystow
import os

# Get environment (development, staging, production)
env = os.getenv("ENVIRONMENT", "development")

def get_env_config(key, **kwargs):
    """Get configuration with environment prefix"""
    return pystow.get_config(f"myapp_{env}", key, **kwargs)

# Environment-specific configuration
api_url = get_env_config("api_url", default="http://localhost:8000")
db_host = get_env_config("db_host", default="localhost") 
debug_mode = get_env_config("debug", dtype=bool, default=True)

Configuration-Driven Downloads

import pystow

def download_configured_dataset(dataset_name):
    """Download dataset using configured URLs"""
    
    # Get base URL from configuration
    base_url = pystow.get_config(
        "datasets", "base_url",
        default="https://data.example.com"
    )
    
    # Get API key if required
    api_key = pystow.get_config("datasets", "api_key", default=None)
    
    # Build download URL
    url = f"{base_url}/datasets/{dataset_name}.csv"
    
    # Configure download parameters
    download_kwargs = {}
    if api_key:
        download_kwargs["headers"] = {"Authorization": f"Bearer {api_key}"}
    
    # Download with configuration
    return pystow.ensure(
        "myapp", "datasets",
        url=url,
        name=f"{dataset_name}.csv",
        download_kwargs=download_kwargs
    )

# Usage
dataset_path = download_configured_dataset("training_data")

Configuration Management Utilities

import pystow

def setup_app_config():
    """Interactive configuration setup"""
    
    print("Setting up MyApp configuration...")
    
    # Get configuration values interactively
    api_key = input("Enter API key: ")
    base_url = input("Enter base URL [https://api.example.com]: ") or "https://api.example.com"
    timeout = input("Enter timeout in seconds [30]: ") or "30"
    
    # Write configuration
    pystow.write_config("myapp", "api_key", api_key)
    pystow.write_config("myapp", "base_url", base_url)
    pystow.write_config("myapp", "timeout", timeout)
    
    print("Configuration saved!")

def validate_config():
    """Validate required configuration exists"""
    
    required_configs = [
        ("myapp", "api_key"),
        ("myapp", "base_url"),
    ]
    
    missing = []
    for module, key in required_configs:
        try:
            pystow.get_config(module, key, raise_on_missing=True)
        except pystow.ConfigError:
            missing.append(f"{module}.{key}")
    
    if missing:
        print(f"Missing required configuration: {', '.join(missing)}")
        return False
    
    print("All required configuration present")
    return True

# Usage
if not validate_config():
    setup_app_config()

Install with Tessl CLI

npx tessl i tessl/pypi-pystow

docs

archives.md

cloud-storage.md

configuration.md

data-formats.md

directory-management.md

file-operations.md

index.md

module-class.md

nltk-integration.md

web-scraping.md

tile.json