A lightweight library for defining and managing system configurations for scientific experimentation.
Overall
score
97%
Evaluation — 97%
↑ 1.05xAgent success when using this tile
{
"context": "This criteria evaluates how well the engineer uses YACS's renamed key management functionality to handle configuration schema evolution. It focuses specifically on the register_renamed_key method and proper error handling during configuration merging.",
"type": "weighted_checklist",
"checklist": [
{
"name": "CfgNode instantiation",
"description": "Creates a CfgNode instance to hold the configuration structure",
"max_score": 10
},
{
"name": "Configuration schema definition",
"description": "Defines the required configuration structure with MODEL.ARCHITECTURE, TRAINING.BATCH_SIZE, and TRAINING.OPTIMIZER using nested CfgNode objects",
"max_score": 15
},
{
"name": "register_renamed_key usage",
"description": "Uses the register_renamed_key method to register both key renames (MODEL.TYPE -> MODEL.ARCHITECTURE and TRAIN.BATCH_SIZE -> TRAINING.BATCH_SIZE)",
"max_score": 25
},
{
"name": "merge_from_list usage",
"description": "Uses the merge_from_list method to apply configuration overrides from the list of key-value pairs",
"max_score": 20
},
{
"name": "Error propagation",
"description": "Allows the KeyError raised by YACS when a renamed key is used to propagate to the caller (does not catch and suppress it)",
"max_score": 15
},
{
"name": "Error message content",
"description": "The raised error message contains both the old key name and new key name (this is provided automatically by YACS's raise_key_rename_error)",
"max_score": 15
}
]
}Install with Tessl CLI
npx tessl i tessl/pypi-yacsdocs
evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10