Database Service Workflow

Build a computational workflow that processes data records using a temporary database service. The workflow should:

1. Start a long-running database service that persists throughout data processing
2. Create a job that loads initial data into the database
3. Process multiple data records in parallel, each querying and updating the database
4. Aggregate results after all processing is complete
5. Ensure the database service stops cleanly after all dependent jobs finish

The database service should provide connection information (host and port) to jobs that need to access it. Jobs should be able to start the service, use it during their execution, and have it automatically cleaned up when no longer needed.

Requirements

• The workflow must use the pipeline framework's service abstraction for long-running processes
• The database service should start after its parent job completes and stop after all child jobs finish
• Connection information must be passed from the service to dependent jobs
• All data processing jobs should run in parallel when possible
• The workflow should handle service lifecycle automatically (start, check, stop)

Test Cases

• Starting the database service returns connection details (host and port) @test
• The service remains running while dependent jobs execute @test
• Multiple parallel jobs can access the same service instance @test
• The service stops automatically after all child jobs complete @test

Implementation

@generates

API

def run_database_workflow(records, job_store_location):
    """
    Execute a workflow that processes records using a database service.

    Args:
        records: List of data records to process
        job_store_location: Path for workflow persistence

    Returns:
        Aggregated results from processing all records
    """
    pass

Dependencies { .dependencies }

toil { .dependency }

Provides workflow management and service job capabilities for managing long-running processes like databases.