A lightweight library for defining and managing system configurations for scientific experimentation.
Overall
score
97%
Evaluation — 97%
↑ 1.05xAgent success when using this tile
{
"context": "This criteria evaluates how well the engineer uses YACS configuration management to merge configuration profiles. The focus is specifically on proper usage of CfgNode objects and the merge_from_other_cfg() method for combining configurations.",
"type": "weighted_checklist",
"checklist": [
{
"name": "CfgNode import",
"description": "Imports CfgNode from yacs.config correctly (typically as 'from yacs.config import CfgNode as CN')",
"max_score": 5
},
{
"name": "CfgNode creation",
"description": "Creates CfgNode instances properly in create_base_config() with nested structure for SYSTEM, TRAIN, and MODEL sections",
"max_score": 15
},
{
"name": "Nested configuration structure",
"description": "Properly creates nested CfgNode objects for hierarchical configuration (e.g., cfg.SYSTEM = CN(), cfg.TRAIN = CN())",
"max_score": 10
},
{
"name": "merge_from_other_cfg usage",
"description": "Uses the merge_from_other_cfg() method to merge profile configurations into the base configuration",
"max_score": 30
},
{
"name": "Correct merge behavior",
"description": "Ensures that merging preserves the base configuration structure while overriding only the specified values from the profile",
"max_score": 20
},
{
"name": "Sequential merging",
"description": "In merge_multiple_profiles(), correctly merges multiple profiles in sequence, with later profiles overriding earlier ones",
"max_score": 15
},
{
"name": "Return values",
"description": "Functions return the modified configuration object appropriately for use in subsequent operations",
"max_score": 5
}
]
}Install with Tessl CLI
npx tessl i tessl/pypi-yacsdocs
evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10