CtrlK
CommunityDocumentationLog inGet started
Tessl Logo

tessl/pypi-toil

Pipeline management software for clusters.

Agent Success

Agent success rate when using this tile

67%

Improvement

Agent success rate improvement when using this tile compared to baseline

1.05x

Baseline

Agent success rate without this tile

64%

Overview
Eval results
Files

rubric.jsonevals/scenario-5/

{
  "context": "This criteria evaluates how well the engineer uses Toil's job store persistence API to initialize, configure, and perform file operations across FileJobStore, AWSJobStore, and GoogleJobStore backends. The focus is on correct usage of Toil's Config, Toil context manager, and job store file management methods.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Toil Initialization",
      "description": "Uses the Toil class constructor or Toil() to initialize job stores with appropriate locator strings for different backends (file paths, aws:region:bucket, google:project:bucket format)",
      "max_score": 20
    },
    {
      "name": "Config Usage",
      "description": "Creates and properly configures Config objects with appropriate settings, including workflow ID when needed",
      "max_score": 15
    },
    {
      "name": "Context Manager Pattern",
      "description": "Uses Toil as a context manager (with statement) or properly calls start() and stop() methods for job store lifecycle management",
      "max_score": 15
    },
    {
      "name": "File Import/Write",
      "description": "Uses appropriate job store methods for writing files (importFile, writeGlobalFile, or similar) to persist files in the job store",
      "max_score": 15
    },
    {
      "name": "File Read/Export",
      "description": "Uses appropriate job store methods for reading files (exportFile, readGlobalFile, or similar) from the job store",
      "max_score": 15
    },
    {
      "name": "File Deletion",
      "description": "Uses deleteGlobalFile or equivalent method to remove files from the job store",
      "max_score": 10
    },
    {
      "name": "Job Store Cleanup",
      "description": "Properly destroys or cleans up job stores using destroyJobStore, Toil.destroy(), or equivalent cleanup methods",
      "max_score": 10
    }
  ]
}
tessl i tessl/pypi-toil@9.0.0

tile.json