tessl/pypi-toil

Pipeline management software for clusters.

Agent Success

Agent success rate when using this tile

67%

Improvement

Agent success rate improvement when using this tile compared to baseline

1.05x

Baseline

Agent success rate without this tile

64%

Overview

Eval results

Files

Workflow Job Store Manager

Name: tessl/pypi-toil
Author: tessl

A utility library for managing Toil workflow job stores across different storage backends. The library provides functions for initializing job stores, performing file operations, and cleaning up resources.

Capabilities

Job Store Initialization

Initialize a job store with a local filesystem path "/tmp/my-jobstore" and return a context that can be used for operations @test
Initialize a job store with an AWS S3 path "aws:us-west-2:my-bucket" and verify it can be used @test
Initialize a job store with a Google Cloud Storage path "google:my-project:my-bucket" and verify it works @test

File Operations

Write a file containing text "Hello World" to an initialized job store, then read it back and verify the content matches @test
Write a file to the job store, delete it, and verify deletion was successful @test

Job Store Cleanup

Initialize a job store, perform operations, then clean up and destroy the job store completely @test

Implementation

@generates

API

def initialize_jobstore(locator: str):
    """
    Initialize a job store based on the locator string.

    Args:
        locator: Job store locator string (file path, AWS path, or GCS path)

    Returns:
        An initialized job store context
    """
    pass

def write_file_to_store(jobstore_context, local_path: str) -> str:
    """
    Write a local file to the job store.

    Args:
        jobstore_context: Initialized job store context
        local_path: Path to local file to write

    Returns:
        File ID in the job store
    """
    pass

def read_file_from_store(jobstore_context, file_id: str, output_path: str):
    """
    Read a file from the job store to a local path.

    Args:
        jobstore_context: Initialized job store context
        file_id: File ID in the job store
        output_path: Local path to write the file
    """
    pass

def delete_file_from_store(jobstore_context, file_id: str):
    """
    Delete a file from the job store.

    Args:
        jobstore_context: Initialized job store context
        file_id: File ID to delete
    """
    pass

def cleanup_jobstore(locator: str):
    """
    Clean up and destroy a job store.

    Args:
        locator: Job store locator
    """
    pass

Dependencies { .dependencies }

toil { .dependency }

Provides workflow management and job store functionality.

@satisfied-by

tessl i tessl/pypi-toil@9.0.0