CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-yacs

A lightweight library for defining and managing system configurations for scientific experimentation.

Overall
score

97%

Evaluation97%

1.05x

Agent success when using this tile

Overview
Eval results
Files

rubric.jsonevals/scenario-5/

{
  "context": "This criteria evaluates how well the engineer uses YACS's CfgNode subclassing capability to implement custom validation logic. The focus is on correctly extending CfgNode, implementing validation in __setattr__, and maintaining proper configuration behavior throughout the class hierarchy.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "CfgNode subclassing",
      "description": "ValidatedConfig class correctly inherits from yacs.config.CfgNode (or CN) using proper Python inheritance syntax",
      "max_score": 20
    },
    {
      "name": "__setattr__ override",
      "description": "Implements custom validation by overriding the __setattr__ method to intercept attribute assignments",
      "max_score": 25
    },
    {
      "name": "Super call preservation",
      "description": "Calls super().__setattr__() to preserve CfgNode's original behavior after validation passes",
      "max_score": 15
    },
    {
      "name": "Learning rate validation",
      "description": "Validates LEARNING_RATE is in range (0.0, 1.0] and raises ValueError with appropriate message when invalid",
      "max_score": 15
    },
    {
      "name": "Batch size validation",
      "description": "Validates BATCH_SIZE is a positive integer and raises ValueError with appropriate message when invalid",
      "max_score": 10
    },
    {
      "name": "Optimizer validation",
      "description": "Validates OPTIMIZER is one of ['adam', 'sgd', 'rmsprop'] and raises ValueError with appropriate message when invalid",
      "max_score": 10
    },
    {
      "name": "Factory function implementation",
      "description": "Implements get_default_config() function that returns a ValidatedConfig instance (not plain CfgNode) with correct default values",
      "max_score": 5
    }
  ]
}

Install with Tessl CLI

npx tessl i tessl/pypi-yacs

tile.json