CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-dagster-aws

Package for AWS-specific Dagster framework solid and resource components.

Pending
Overview
Eval results
Files

secrets-management.mddocs/

Secrets Management

Integrate with AWS Secrets Manager for secure credential and configuration management within Dagster pipelines.

Capabilities

Secrets Manager Resource

class SecretsManagerResource(ResourceWithBoto3Configuration):
    """
    Resource for accessing AWS Secrets Manager.
    """
    
    def get_secret(self, secret_id: str) -> Dict:
        """
        Retrieve secret value from Secrets Manager.
        
        Parameters:
            secret_id: Secret ARN or name
            
        Returns:
            Dict: Secret value and metadata
        """
    
    def get_secret_string(self, secret_id: str) -> str:
        """
        Get secret as string value.
        """

class SecretsManagerSecretsResource(ConfigurableResource):
    """
    Resource providing secrets as configuration.
    """
    secrets_manager: SecretsManagerResource
    secret_arns: List[str]

def secretsmanager_resource(**kwargs) -> SecretsManagerResource: ...
def secretsmanager_secrets_resource(**kwargs): ...

Secrets Utilities

def get_secrets_from_arns(arns: List[str]) -> Dict[str, Any]:
    """
    Retrieve multiple secrets by ARN.
    
    Parameters:
        arns: List of secret ARNs
        
    Returns:
        Dict: Mapping of ARN to secret value
    """

def get_tagged_secrets(tags: Dict[str, str]) -> Dict[str, Any]:
    """
    Retrieve secrets by tags.
    
    Parameters:
        tags: Tag filters
        
    Returns:
        Dict: Secrets matching tags
    """

Usage Examples

from dagster import op, job, Definitions
from dagster_aws.secretsmanager import SecretsManagerResource

@op(required_resource_keys={"secrets"})
def connect_to_database(context):
    secrets = context.resources.secrets
    
    # Retrieve database credentials
    db_secret = secrets.get_secret("prod/database/credentials")
    credentials = json.loads(db_secret['SecretString'])
    
    # Use credentials to connect
    connection = connect_db(
        host=credentials['host'],
        username=credentials['username'], 
        password=credentials['password']
    )
    
    return connection

@job(
    resource_defs={
        "secrets": SecretsManagerResource(region_name="us-west-2")
    }
)
def secure_job():
    connect_to_database()

defs = Definitions(jobs=[secure_job])

Install with Tessl CLI

npx tessl i tessl/pypi-dagster-aws

docs

athena-queries.md

cloudwatch-logging.md

ecr-integration.md

ecs-orchestration.md

emr-processing.md

index.md

parameter-store.md

pipes-orchestration.md

rds-operations.md

redshift-integration.md

s3-storage.md

secrets-management.md

tile.json