CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-multitasking

Non-blocking Python methods using decorators

Pending
Overview
Eval results
Files

pool-management.mddocs/

Pool Management

Create and manage execution pools with different configurations for various types of workloads. Pools control how many tasks can run concurrently and which execution engine to use.

Capabilities

Pool Creation

Create new execution pools with custom thread counts and engine types.

def createPool(name: str = "main", threads: Optional[int] = None, engine: Optional[str] = None) -> None:
    """
    Create a new execution pool with specified configuration.
    
    Pools manage concurrent task execution using semaphores. Each pool
    has its own thread/process limit and engine type. Creating a pool
    automatically makes it the active pool for new tasks.
    
    Args:
        name: Unique identifier for this pool
        threads: Max concurrent tasks. None uses global MAX_THREADS.
                 Values < 2 create unlimited pools (no semaphore).
        engine: "process" or "thread". None uses global ENGINE setting.
    
    Note:
        Setting threads=0 or threads=1 creates an unlimited pool where
        all tasks run immediately without queuing.
    """

Usage Example:

import multitasking

# Create different pools for different workloads
multitasking.createPool("api_calls", threads=20, engine="thread")
multitasking.createPool("cpu_intensive", threads=4, engine="process")
multitasking.createPool("unlimited", threads=0, engine="thread")

# The most recently created pool becomes active
@multitasking.task
def process_data(data):
    # This will use the "unlimited" pool
    pass

# Tasks will use whatever pool was active when they were defined

Pool Information Retrieval

Get information about existing execution pools.

def getPool(name: Optional[str] = None) -> Dict[str, Union[str, int]]:
    """
    Retrieve information about an execution pool.
    
    Returns a dictionary with pool metadata including engine type,
    name, and thread count. Useful for debugging and monitoring.
    
    Args:
        name: Pool name to query. If None, uses current active pool.
    
    Returns:
        Dictionary with keys: 'engine', 'name', 'threads'
    
    Raises:
        KeyError: If the specified pool doesn't exist
    """

Usage Example:

import multitasking

# Create some pools
multitasking.createPool("api_pool", threads=10, engine="thread")
multitasking.createPool("cpu_pool", threads=2, engine="process")

# Get information about specific pools
api_info = multitasking.getPool("api_pool")
print(f"API Pool: {api_info}")
# Output: {'engine': 'thread', 'name': 'api_pool', 'threads': 10}

cpu_info = multitasking.getPool("cpu_pool")
print(f"CPU Pool: {cpu_info}")
# Output: {'engine': 'process', 'name': 'cpu_pool', 'threads': 2}

# Get current active pool info
current_info = multitasking.getPool()
print(f"Current Pool: {current_info}")

Pool Usage Patterns

Specialized Workloads

Create different pools for different types of operations:

import multitasking
import requests
import time

# Fast pool for quick API calls
multitasking.createPool("fast_api", threads=50, engine="thread")

@multitasking.task
def fetch_url(url):
    response = requests.get(url, timeout=5)
    return response.status_code

# Switch to CPU pool for intensive work
multitasking.createPool("compute", threads=2, engine="process")

@multitasking.task
def heavy_computation(data):
    # CPU-intensive processing
    return sum(x*x for x in data)

# Use the appropriate pool for each task type
urls = ["http://example.com" for _ in range(10)]
for url in urls:
    fetch_url(url)  # Uses compute pool (last created)

# Switch back to api pool by recreating it
multitasking.createPool("fast_api", threads=50, engine="thread")

Dynamic Pool Configuration

Adjust pool settings based on runtime conditions:

import multitasking

# Start with conservative settings
multitasking.createPool("dynamic", threads=2, engine="thread")

def adjust_pool_size(load_factor):
    if load_factor > 0.8:
        # High load: increase parallelism
        multitasking.createPool("dynamic", threads=20, engine="thread")
    elif load_factor < 0.3:
        # Low load: reduce resource usage
        multitasking.createPool("dynamic", threads=5, engine="thread")

# Monitor and adjust
current_load = 0.9  # Example load metric
adjust_pool_size(current_load)

pool_info = multitasking.getPool("dynamic")
print(f"Adjusted to: {pool_info['threads']} threads")

Install with Tessl CLI

npx tessl i tessl/pypi-multitasking

docs

configuration.md

index.md

monitoring-control.md

pool-management.md

task-management.md

tile.json