Batch Task Manager with Constraints

A Python application that submits batch processing tasks to Azure Batch with proper time constraints and retention policies.

Background

Your team needs to process video files in the cloud using Azure Batch. Each video processing task has specific time limits and retry requirements. Additionally, task results must be retained for different durations based on their success or failure status.

Requirements

Create a module that submits tasks to an Azure Batch job with the following constraints:

1. Time Constraints: Tasks must have a maximum wall-clock time limit to prevent runaway processes
2. Retry Behavior: Failed tasks should automatically retry with a configurable maximum retry count
3. Retention Policy: Task data must be retained for a specified duration after completion to allow for result retrieval and debugging

The module should accept task configuration parameters and create tasks with appropriate constraint settings.

Capabilities

Task Creation with Wall-Clock Time Limit

• Given a task command and a wall-clock time limit of 3600 seconds, creates a task with max_wall_clock_time set to 3600 seconds @test

Task Creation with Retry Configuration

• Given a task command and a max retry count of 3, creates a task with max_task_retry_count set to 3 @test

Task Creation with Retention Time

• Given a task command and a retention time of 7 days, creates a task with retention_time set to 7 days @test

Task Creation with All Constraints

• Given a task command with wall-clock limit of 1800 seconds, max retry count of 2, and retention time of 3 days, creates a task with all three constraint parameters properly configured @test

Implementation

@generates

API

from datetime import timedelta
from azure.batch.models import TaskAddParameter, TaskConstraints

def create_task_with_constraints(
    task_id: str,
    command_line: str,
    max_wall_clock_time: timedelta = None,
    max_retry_count: int = None,
    retention_time: timedelta = None
) -> TaskAddParameter:
    """
    Creates a TaskAddParameter with specified constraints.

    Args:
        task_id: Unique identifier for the task
        command_line: Command line to execute
        max_wall_clock_time: Maximum wall-clock time for task execution
        max_retry_count: Maximum number of retry attempts
        retention_time: Duration to retain task data after completion

    Returns:
        TaskAddParameter object configured with the specified constraints
    """
    pass

Dependencies { .dependencies }

azure-batch { .dependency }

Provides Azure Batch client library for task management and constraint configuration.

@satisfied-by