A lightweight library for defining and managing system configurations for scientific experimentation.
Overall
score
97%
Evaluation — 97%
↑ 1.05xAgent success when using this tile
{
"context": "This criteria evaluates how well the engineer uses YACS's dynamic key addition control (NEW_ALLOWED) feature to implement a plugin configuration system with different validation levels for different configuration sections.",
"type": "weighted_checklist",
"checklist": [
{
"name": "CfgNode initialization",
"description": "Creates configuration using CfgNode (or imported as CN) from yacs.config for the main config and its sections (CORE, PLUGINS, USER_PREFS)",
"max_score": 15
},
{
"name": "PLUGINS new_allowed=True",
"description": "Initializes PLUGINS section with new_allowed=True parameter (e.g., CN(new_allowed=True)) to allow arbitrary plugin keys",
"max_score": 25
},
{
"name": "CORE strict validation",
"description": "CORE section uses default new_allowed=False behavior (or explicitly sets it to False) to prevent undefined keys from being added",
"max_score": 15
},
{
"name": "merge_from_list usage",
"description": "Uses merge_from_list() method to load plugin configurations from the key-value list format",
"max_score": 15
},
{
"name": "is_new_allowed() check",
"description": "Uses is_new_allowed() method to check whether a configuration section accepts new keys",
"max_score": 15
},
{
"name": "set_new_allowed() toggle",
"description": "Uses set_new_allowed(True) method to enable dynamic key addition for the USER_PREFS section",
"max_score": 15
}
]
}Install with Tessl CLI
npx tessl i tessl/pypi-yacsdocs
evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10