Distributed Matrix Multiplication Coordinator

A distributed computing task coordinator that performs matrix multiplication across multiple compute nodes using parallel processing.

Problem Description

You need to implement a system that coordinates a distributed matrix multiplication operation across multiple compute nodes in a cloud environment. The system should split the work across multiple nodes, where one node acts as the primary coordinator and other nodes perform subtasks.

The coordinator should:

1. Set up a distributed task that runs across multiple compute nodes
2. Configure the primary task to coordinate the computation
3. Configure subtasks to run on worker nodes
4. Ensure all nodes can communicate with each other
5. Share necessary input files across all task instances

Requirements

• The distributed task must run across at least 3 compute nodes
• One node should act as the primary coordinator
• Worker nodes should execute a subtask command
• All nodes must be able to communicate with each other for data exchange
• Input data files should be available to all task instances
• The implementation should handle coordination settings appropriately

Implementation

@generates

API

def create_multi_instance_task(
    task_id: str,
    job_id: str,
    pool_id: str,
    num_instances: int,
    coordination_command: str,
    worker_command: str,
    resource_files: list
) -> dict:
    """
    Creates a multi-instance task configuration for distributed matrix multiplication.

    Args:
        task_id: Unique identifier for the task
        job_id: ID of the job this task belongs to
        pool_id: ID of the compute pool
        num_instances: Number of compute nodes to use (minimum 3)
        coordination_command: Command line for the primary coordinator task
        worker_command: Command line for worker subtasks
        resource_files: List of input files to share across all instances

    Returns:
        A dictionary containing the multi-instance task configuration with:
        - Task ID and basic settings
        - Multi-instance configuration with number of instances
        - Coordination command for primary task
        - Worker command for subtasks
        - Resource files for all instances
        - Inter-node communication settings enabled

    Raises:
        ValueError: If num_instances < 3
    """
    pass

Test Cases

Multi-Instance Configuration

• Creating a task with 3 instances returns a configuration with num_instances set to 3 @test
• Creating a task with 2 instances raises ValueError @test

Task Commands

• The returned configuration includes the coordination_command for the primary task @test
• The returned configuration includes the worker_command for subtasks @test

Resource Sharing

• Resource files are included in the multi-instance configuration @test

Dependencies { .dependencies }

azure-batch { .dependency }

Provides Azure Batch client functionality for creating and managing distributed computing tasks across multiple nodes.

@satisfied-by