CtrlK
CommunityDocumentationLog inGet started
Tessl Logo

tessl/pypi-toil

Pipeline management software for clusters.

Agent Success

Agent success rate when using this tile

67%

Improvement

Agent success rate improvement when using this tile compared to baseline

1.05x

Baseline

Agent success rate without this tile

64%

Overview
Eval results
Files

rubric.jsonevals/scenario-4/

{
  "context": "This evaluation assesses how effectively the engineer uses Toil's container support APIs to configure Docker and Singularity container execution in workflows. The focus is on proper usage of Toil's configuration properties for enabling container runtimes and setting container-specific parameters.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Docker support enabled",
      "description": "Uses config.enableCWLDockerSupport or equivalent property to enable Docker container execution",
      "max_score": 30
    },
    {
      "name": "Singularity support enabled",
      "description": "Uses config.cwlUseSingularity or equivalent property to enable Singularity as the container runtime",
      "max_score": 25
    },
    {
      "name": "Singularity arguments configured",
      "description": "Uses config.singularityArgs to set runtime arguments (e.g., ['--cleanenv', '--containall']) for Singularity execution",
      "max_score": 25
    },
    {
      "name": "Container image set",
      "description": "Uses config.dockerAppliance or equivalent property to specify the container image to use for execution",
      "max_score": 20
    }
  ]
}
tessl i tessl/pypi-toil@9.0.0

tile.json