CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-apache-libcloud

A standard Python library that abstracts away differences among multiple cloud provider APIs

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Overview
Eval results
Files

container-services.mddocs/

Container Services

The container service provides a unified interface for container and cluster management across multiple container platforms including Kubernetes, Docker, AWS ECS, Google Container Engine, Azure Container Service, and others.

Providers

from libcloud.container.types import Provider

class Provider:
    """Enumeration of supported container providers"""
    DOCKER = 'docker'                    # Docker Engine
    KUBERNETES = 'kubernetes'            # Kubernetes clusters
    ECS = 'ecs'                         # AWS Elastic Container Service
    GKE = 'gke'                         # Google Kubernetes Engine
    AKS = 'aks'                         # Azure Kubernetes Service
    RANCHERK8S = 'rancherk8s'           # Rancher Kubernetes
    LXD = 'lxd'                         # LXD containers
    DIMENSIONDATA = 'dimensiondata'      # Dimension Data containers
    JOYENT = 'joyent'                   # Joyent Triton
    # ... more providers

Driver Factory

from libcloud.container.providers import get_driver

def get_driver(provider: Provider) -> type[ContainerDriver]

Get the driver class for a specific container provider.

Parameters:

  • provider: Provider identifier from the Provider enum

Returns:

  • Driver class for the specified provider

Example:

from libcloud.container.types import Provider
from libcloud.container.providers import get_driver

# Get Kubernetes driver class
cls = get_driver(Provider.KUBERNETES)

# Initialize driver with credentials
driver = cls('api_server_url', key='api_token')

Core Classes

ContainerDriver

class ContainerDriver(BaseDriver):
    """Base class for all container drivers"""
    
    def list_containers(self, image: ContainerImage = None, all: bool = True) -> List[Container]
    def get_container(self, id: str) -> Container
    def start_container(self, container: Container) -> Container
    def stop_container(self, container: Container) -> Container
    def restart_container(self, container: Container) -> Container
    def destroy_container(self, container: Container) -> bool
    def deploy_container(self, name: str, image: ContainerImage, cluster: ContainerCluster = None, **kwargs) -> Container
    def list_images(self) -> List[ContainerImage]
    def install_image(self, path: str) -> ContainerImage
    def list_clusters(self) -> List[ContainerCluster]
    def get_cluster(self, id: str) -> ContainerCluster
    def create_cluster(self, name: str, **kwargs) -> ContainerCluster
    def destroy_cluster(self, cluster: ContainerCluster) -> bool
    def ex_list_cluster_nodes(self, cluster: ContainerCluster) -> List[Dict]

Base class that all container drivers inherit from. Provides methods for managing containers, images, and clusters.

Key Methods:

  • list_containers(): List all containers
  • deploy_container(): Deploy a new container
  • start_container(): Start a stopped container
  • stop_container(): Stop a running container
  • destroy_container(): Remove a container
  • list_clusters(): List container clusters
  • create_cluster(): Create a new cluster

Container

class Container:
    """Represents a container instance"""
    
    id: str
    name: str
    image: ContainerImage
    state: ContainerState
    ip_addresses: List[str]
    driver: ContainerDriver
    extra: Dict[str, Any]
    
    def start(self) -> Container
    def stop(self) -> Container
    def restart(self) -> Container  
    def destroy(self) -> bool
    def get_logs(self) -> str

Represents a container instance.

Properties:

  • id: Unique container identifier
  • name: Container name
  • image: Container image used
  • state: Current state (running, stopped, pending, etc.)
  • ip_addresses: List of IP addresses assigned to container
  • extra: Provider-specific metadata (ports, volumes, environment variables, etc.)

Methods:

  • start(): Start this container
  • stop(): Stop this container
  • restart(): Restart this container
  • destroy(): Remove this container
  • get_logs(): Get container logs

ContainerImage

class ContainerImage:
    """Represents a container image"""
    
    id: str
    name: str
    tag: str
    driver: ContainerDriver
    version: str
    extra: Dict[str, Any]
    
    def deploy(self, name: str, cluster: ContainerCluster = None, **kwargs) -> Container

Represents a container image that can be used to create containers.

Properties:

  • id: Unique image identifier
  • name: Image name (e.g., "nginx", "ubuntu")
  • tag: Image tag/version (e.g., "latest", "1.21")
  • version: Image version
  • extra: Provider-specific metadata (size, created date, etc.)

Methods:

  • deploy(): Deploy a container from this image

ContainerCluster

class ContainerCluster:
    """Represents a container cluster"""
    
    id: str
    name: str
    driver: ContainerDriver
    state: str
    extra: Dict[str, Any]
    
    def list_containers(self) -> List[Container]
    def deploy_container(self, name: str, image: ContainerImage, **kwargs) -> Container
    def destroy(self) -> bool

Represents a container cluster (e.g., Kubernetes cluster, ECS cluster).

Properties:

  • id: Unique cluster identifier
  • name: Cluster name
  • state: Current state (active, creating, deleting, etc.)
  • extra: Provider-specific metadata (node count, version, etc.)

Methods:

  • list_containers(): List containers in this cluster
  • deploy_container(): Deploy a container to this cluster
  • destroy(): Delete this cluster

ContainerState

class ContainerState:
    """Container states enumeration"""
    RUNNING = 0
    STOPPED = 1
    PENDING = 2
    TERMINATED = 3
    UNKNOWN = 4
    ERROR = 5
    PAUSED = 6
    SUSPENDED = 7
    STARTING = 8
    STOPPING = 9

Enumeration of possible container states during their lifecycle.

Docker Registry Support

from libcloud.container.utils.docker import RegistryClient, HubClient

class RegistryClient:
    """Docker registry client"""
    
    def __init__(self, host: str = 'registry-1.docker.io', port: int = 443, username: str = None, password: str = None)
    def get_image(self, name: str, tag: str = 'latest') -> Dict
    def list_images(self) -> List[Dict]
    def push_image(self, name: str, tag: str = 'latest') -> bool
    def pull_image(self, name: str, tag: str = 'latest') -> bool

class HubClient(RegistryClient):
    """Docker Hub client"""
    
    def __init__(self, username: str = None, password: str = None)
    def get_repository(self, name: str) -> Dict
    def list_repositories(self, username: str = None) -> List[Dict]

Utilities for interacting with Docker registries and Docker Hub.

Usage Examples

Basic Container Management

from libcloud.container.types import Provider, ContainerState
from libcloud.container.providers import get_driver

# Initialize Docker driver
cls = get_driver(Provider.DOCKER)
driver = cls(host='unix:///var/run/docker.sock')

# Alternative: Remote Docker daemon
# driver = cls(host='tcp://docker-host:2376', cert_path='/path/to/certs')

# List existing containers
containers = driver.list_containers(all=True)  # Include stopped containers
for container in containers:
    print(f"Container: {container.name} ({container.state}) - Image: {container.image.name}")

# List available images
images = driver.list_images()
for image in images:
    print(f"Image: {image.name}:{image.tag} (ID: {image.id[:12]})")

# Deploy a new container
nginx_image = None
for image in images:
    if 'nginx' in image.name:
        nginx_image = image
        break

if nginx_image:
    container = driver.deploy_container(
        name='web-server',
        image=nginx_image,
        ports=[80, 443],
        environment={'ENV': 'production'},
        volumes={'/var/www/html': '/usr/share/nginx/html'}
    )
    print(f"Deployed container: {container.name} ({container.id})")

Kubernetes Container Management

# Initialize Kubernetes driver
k8s_cls = get_driver(Provider.KUBERNETES)
k8s_driver = k8s_cls(
    host='https://k8s-api-server:6443',
    key='api_token',  # Or use cert_file/key_file for certificate auth
    ca_cert='/path/to/ca.crt'
)

# List clusters (nodes in Kubernetes context)
clusters = k8s_driver.list_clusters()
for cluster in clusters:
    print(f"Cluster: {cluster.name} ({cluster.state})")
    if cluster.extra:
        print(f"  Nodes: {cluster.extra.get('node_count', 'unknown')}")

# List pods (containers in Kubernetes)
pods = k8s_driver.list_containers()
for pod in pods:
    namespace = pod.extra.get('namespace', 'default')
    print(f"Pod: {pod.name} in namespace {namespace} ({pod.state})")
    print(f"  Image: {pod.image.name}")
    print(f"  IPs: {pod.ip_addresses}")

# Deploy a pod
pod = k8s_driver.deploy_container(
    name='web-app',
    image=ContainerImage(id=None, name='nginx', tag='1.21', driver=k8s_driver),
    ex_namespace='production',
    ex_replicas=3,
    ex_service_ports=[{'port': 80, 'target_port': 80}]
)
print(f"Deployed pod: {pod.name}")

AWS ECS Container Management

# Initialize ECS driver
ecs_cls = get_driver(Provider.ECS)
ecs_driver = ecs_cls('access_key', 'secret_key', region='us-east-1')

# List ECS clusters
clusters = ecs_driver.list_clusters()
for cluster in clusters:
    print(f"ECS Cluster: {cluster.name}")
    print(f"  Active Services: {cluster.extra.get('active_services_count', 0)}")
    print(f"  Running Tasks: {cluster.extra.get('running_tasks_count', 0)}")

# Create a new ECS cluster
new_cluster = ecs_driver.create_cluster(
    name='production-cluster',
    ex_capacity_providers=['FARGATE', 'EC2']
)
print(f"Created ECS cluster: {new_cluster.name}")

# Deploy a task (container) to ECS
task = ecs_driver.deploy_container(
    name='web-service',
    image=ContainerImage(id=None, name='nginx', tag='latest', driver=ecs_driver),
    cluster=new_cluster,
    ex_cpu=256,        # 0.25 vCPU
    ex_memory=512,     # 512 MB
    ex_network_mode='awsvpc',
    ex_launch_type='FARGATE'
)
print(f"Deployed ECS task: {task.name}")

Container Lifecycle Management

# Get a specific container
container = driver.get_container('container-id-123')
print(f"Container state: {container.state}")

# Start a stopped container
if container.state == ContainerState.STOPPED:
    started_container = container.start()
    print(f"Started container: {started_container.state}")

# Stop a running container
if container.state == ContainerState.RUNNING:
    stopped_container = container.stop()
    print(f"Stopped container: {stopped_container.state}")

# Restart a container
restarted_container = container.restart()
print(f"Restarted container: {restarted_container.state}")

# Get container logs
try:
    logs = container.get_logs()
    print(f"Container logs:\n{logs}")
except AttributeError:
    print("Log retrieval not supported by this provider")

# Remove a container
success = container.destroy()
print(f"Container removed: {success}")

Advanced Container Deployment

def deploy_multi_tier_application(driver, cluster=None):
    """Deploy a multi-tier application with database and web components"""
    
    # Database container
    db_container = driver.deploy_container(
        name='app-database',
        image=ContainerImage(id=None, name='postgres', tag='13', driver=driver),
        cluster=cluster,
        environment={
            'POSTGRES_DB': 'appdb',
            'POSTGRES_USER': 'appuser',
            'POSTGRES_PASSWORD': 'secure_password'
        },
        volumes={'/var/lib/postgresql/data': '/data/postgres'},
        ex_restart_policy='always'
    )
    print(f"Deployed database: {db_container.name}")
    
    # Web application container
    web_container = driver.deploy_container(
        name='app-web',
        image=ContainerImage(id=None, name='myapp', tag='v1.0', driver=driver),
        cluster=cluster,
        ports=[8080],
        environment={
            'DATABASE_URL': f'postgresql://appuser:secure_password@{db_container.name}:5432/appdb',
            'APP_ENV': 'production'
        },
        ex_links=[db_container.name],  # Link to database container
        ex_restart_policy='unless-stopped'
    )
    print(f"Deployed web app: {web_container.name}")
    
    # Load balancer/reverse proxy container
    proxy_container = driver.deploy_container(
        name='app-proxy',
        image=ContainerImage(id=None, name='nginx', tag='alpine', driver=driver),
        cluster=cluster,
        ports=[80, 443],
        volumes={'/etc/nginx/nginx.conf': '/config/nginx.conf'},
        ex_links=[web_container.name],
        ex_restart_policy='always'
    )
    print(f"Deployed proxy: {proxy_container.name}")
    
    return [db_container, web_container, proxy_container]

# Deploy the application
app_containers = deploy_multi_tier_application(driver)

Docker Registry Integration

from libcloud.container.utils.docker import RegistryClient, HubClient

# Connect to Docker Hub
hub_client = HubClient(username='myusername', password='mypassword')

# List repositories
repos = hub_client.list_repositories('myusername')
for repo in repos:
    print(f"Repository: {repo['name']} - Stars: {repo.get('star_count', 0)}")

# Get repository details
repo_info = hub_client.get_repository('myusername/myapp')
print(f"Repository: {repo_info['name']}")
print(f"Description: {repo_info.get('description', 'No description')}")
print(f"Last updated: {repo_info.get('last_updated')}")

# Connect to private registry
registry_client = RegistryClient(
    host='my-registry.company.com',
    port=443,
    username='registry_user',
    password='registry_password'
)

# Pull image from private registry
success = registry_client.pull_image('private-app', 'v2.1')
print(f"Image pulled: {success}")

# Push image to private registry
success = registry_client.push_image('my-app', 'latest')
print(f"Image pushed: {success}")

Container Monitoring and Health Checks

import time
from typing import List, Dict

def monitor_container_health(driver, containers: List[Container], interval: int = 30):
    """Monitor container health and restart unhealthy containers"""
    
    print(f"Monitoring {len(containers)} containers...")
    
    while True:
        try:
            for container in containers:
                # Refresh container state
                current_container = driver.get_container(container.id)
                
                print(f"Container {current_container.name}: {current_container.state}")
                
                # Check if container should be running but isn't
                if (current_container.state in [ContainerState.STOPPED, ContainerState.ERROR] and 
                    container.extra.get('expected_state') == 'running'):
                    
                    print(f"Restarting unhealthy container: {current_container.name}")
                    restarted = current_container.restart()
                    print(f"Restart result: {restarted.state}")
                
                # Check resource usage if available
                if 'resource_usage' in current_container.extra:
                    usage = current_container.extra['resource_usage']
                    cpu_percent = usage.get('cpu_percent', 0)
                    memory_percent = usage.get('memory_percent', 0)
                    
                    print(f"  CPU: {cpu_percent:.1f}%, Memory: {memory_percent:.1f}%")
                    
                    # Alert on high resource usage
                    if cpu_percent > 90 or memory_percent > 90:
                        print(f"WARNING: High resource usage in {current_container.name}")
        
        except Exception as e:
            print(f"Error monitoring containers: {e}")
        
        time.sleep(interval)

def get_container_stats(driver, container: Container) -> Dict:
    """Get container resource statistics"""
    try:
        # This is provider-specific - Docker example
        if hasattr(driver, 'ex_get_container_stats'):
            stats = driver.ex_get_container_stats(container)
            return {
                'cpu_usage': stats.get('cpu_usage', {}),
                'memory_usage': stats.get('memory_usage', {}),
                'network_io': stats.get('network_io', {}),
                'block_io': stats.get('block_io', {})
            }
    except AttributeError:
        pass
    
    return {}

# Usage
important_containers = [container for container in driver.list_containers() 
                       if container.state == ContainerState.RUNNING]

# Get stats for each container
for container in important_containers:
    stats = get_container_stats(driver, container)
    if stats:
        print(f"Stats for {container.name}: {stats}")

# Start monitoring (run in separate thread/process)
# monitor_container_health(driver, important_containers, interval=60)

Container Scaling and Load Balancing

def scale_container_service(driver, service_name: str, target_count: int, cluster: ContainerCluster = None):
    """Scale a container service up or down"""
    
    # Get current containers for the service
    all_containers = driver.list_containers()
    service_containers = [c for c in all_containers if service_name in c.name]
    
    current_count = len(service_containers)
    print(f"Current {service_name} containers: {current_count}, Target: {target_count}")
    
    if current_count < target_count:
        # Scale up
        to_create = target_count - current_count
        print(f"Scaling up: creating {to_create} new containers")
        
        # Find a template container to replicate
        template = service_containers[0] if service_containers else None
        if not template:
            print("No template container found for scaling")
            return
        
        for i in range(to_create):
            new_container = driver.deploy_container(
                name=f"{service_name}-{current_count + i + 1}",
                image=template.image,
                cluster=cluster,
                ports=template.extra.get('ports', []),
                environment=template.extra.get('environment', {}),
                volumes=template.extra.get('volumes', {})
            )
            print(f"Created container: {new_container.name}")
    
    elif current_count > target_count:
        # Scale down
        to_remove = current_count - target_count
        print(f"Scaling down: removing {to_remove} containers")
        
        # Remove excess containers (oldest first)
        containers_to_remove = service_containers[:to_remove]
        for container in containers_to_remove:
            success = container.destroy()
            print(f"Removed container {container.name}: {success}")
    
    else:
        print("Service is already at target scale")

def create_container_load_balancer(driver, service_containers: List[Container], lb_port: int = 80):
    """Create a load balancer container for a service"""
    
    # Generate nginx configuration for load balancing
    upstream_servers = []
    for container in service_containers:
        if container.ip_addresses:
            ip = container.ip_addresses[0]
            port = container.extra.get('exposed_ports', [8080])[0]
            upstream_servers.append(f"server {ip}:{port};")
    
    nginx_config = f"""
upstream backend {{
    {chr(10).join(upstream_servers)}
}}

server {{
    listen {lb_port};
    location / {{
        proxy_pass http://backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }}
}}
"""
    
    # Deploy load balancer container
    lb_container = driver.deploy_container(
        name='service-load-balancer',
        image=ContainerImage(id=None, name='nginx', tag='alpine', driver=driver),
        ports=[lb_port],
        volumes={'/etc/nginx/conf.d/default.conf': nginx_config},
        ex_restart_policy='always'
    )
    
    print(f"Created load balancer: {lb_container.name}")
    return lb_container

# Usage
# Scale web service to 5 containers
scale_container_service(driver, 'web-app', 5)

# Create load balancer for web service containers
web_containers = [c for c in driver.list_containers() if 'web-app' in c.name]
if web_containers:
    lb = create_container_load_balancer(driver, web_containers, 80)

Multi-Provider Container Management

from libcloud.container.types import Provider
from libcloud.container.providers import get_driver

# Configure multiple container providers
container_providers = {
    'docker_local': {
        'driver': get_driver(Provider.DOCKER),
        'config': {'host': 'unix:///var/run/docker.sock'}
    },
    'kubernetes': {
        'driver': get_driver(Provider.KUBERNETES),
        'config': {
            'host': 'https://k8s-cluster:6443',
            'key': 'api_token'
        }
    },
    'ecs': {
        'driver': get_driver(Provider.ECS),
        'config': {
            'key': 'aws_access_key',
            'secret': 'aws_secret_key',
            'region': 'us-east-1'
        }
    }
}

# Initialize drivers
container_drivers = {}
for name, config in container_providers.items():
    cls = config['driver']  
    if name == 'docker_local':
        container_drivers[name] = cls(**config['config'])
    elif name == 'kubernetes':
        container_drivers[name] = cls(**config['config'])
    elif name == 'ecs':
        container_drivers[name] = cls(
            config['config']['key'],
            config['config']['secret'],
            region=config['config']['region']
        )

# Deploy the same application across providers
def deploy_across_providers(app_config):
    """Deploy application across multiple container providers"""
    deployments = {}
    
    for provider_name, driver in container_drivers.items():
        try:
            print(f"Deploying to {provider_name}...")
            
            container = driver.deploy_container(
                name=f"{app_config['name']}-{provider_name}",
                image=ContainerImage(
                    id=None,
                    name=app_config['image'],
                    tag=app_config['tag'],
                    driver=driver
                ),
                ports=app_config['ports'],
                environment=app_config['environment']
            )
            
            deployments[provider_name] = container
            print(f"Deployed to {provider_name}: {container.name}")
            
        except Exception as e:
            print(f"Failed to deploy to {provider_name}: {e}")
    
    return deployments

# Application configuration
app_config = {
    'name': 'web-api',
    'image': 'nginx',
    'tag': 'alpine',
    'ports': [80],
    'environment': {'ENV': 'production'}
}

# Deploy across all providers
multi_deployments = deploy_across_providers(app_config)

Exception Handling

from libcloud.container.types import ContainerError
from libcloud.common.types import LibcloudError, InvalidCredsError

try:
    # Deploy container
    container = driver.deploy_container(
        name='test-container',
        image=ContainerImage(id=None, name='nginx', tag='latest', driver=driver)
    )
except InvalidCredsError:
    print("Invalid credentials for container provider")
except ContainerError as e:
    print(f"Container specific error: {e}")
except LibcloudError as e:
    print(f"General Libcloud error: {e}")

# Check container state before operations
container = driver.get_container('container-123')
if container.state == ContainerState.RUNNING:
    success = container.stop()
elif container.state == ContainerState.STOPPED:
    success = container.start()

Provider-Specific Features

Different providers offer additional features through the ex_* parameter pattern:

# Docker specific features
docker_driver = get_driver(Provider.DOCKER)('unix:///var/run/docker.sock')

# Deploy with Docker-specific options
container = docker_driver.deploy_container(
    name='advanced-container',
    image=ContainerImage(id=None, name='nginx', tag='latest', driver=docker_driver),
    ex_privileged=True,           # Run in privileged mode
    ex_network_mode='host',       # Use host networking
    ex_restart_policy='always',   # Always restart
    ex_cpu_shares=512,           # CPU priority
    ex_memory_limit='1g'         # Memory limit
)

# Kubernetes specific features  
k8s_driver = get_driver(Provider.KUBERNETES)('https://k8s-api:6443', key='token')

# Deploy with Kubernetes-specific options
pod = k8s_driver.deploy_container(
    name='k8s-app',
    image=ContainerImage(id=None, name='nginx', tag='1.21', driver=k8s_driver),
    ex_namespace='production',           # Kubernetes namespace
    ex_replicas=3,                      # Number of replicas
    ex_service_type='LoadBalancer',     # Service type
    ex_resource_requests={'cpu': '100m', 'memory': '128Mi'},  # Resource requests
    ex_resource_limits={'cpu': '500m', 'memory': '512Mi'}     # Resource limits
)

# ECS specific features
ecs_driver = get_driver(Provider.ECS)('access_key', 'secret_key', region='us-east-1')

# Deploy with ECS-specific options
task = ecs_driver.deploy_container(
    name='ecs-task',
    image=ContainerImage(id=None, name='nginx', tag='latest', driver=ecs_driver),
    ex_task_definition_arn='arn:aws:ecs:us-east-1:123456789012:task-definition/my-task:1',
    ex_launch_type='FARGATE',           # Launch type
    ex_network_configuration={          # Network configuration for Fargate
        'awsvpcConfiguration': {
            'subnets': ['subnet-12345'],
            'securityGroups': ['sg-12345'],
            'assignPublicIp': 'ENABLED'
        }
    }
)

Check provider-specific documentation for additional capabilities available through the ex_* parameters and methods.

Install with Tessl CLI

npx tessl i tessl/pypi-apache-libcloud

docs

backup-services.md

compute-services.md

container-services.md

core-driver-system.md

dns-management.md

index.md

load-balancer-services.md

storage-services.md

tile.json