General-purpose coding policy for Baruch's AI agents
90
91%
Does it follow best practices?
Impact
90%
1.30xAverage score across 18 eval scenarios
Advisory
Suggest reviewing before use
{
"context": "Tests whether the agent correctly resists path-scoping a rule whose prescriptions span multiple contexts. The rule has two halves: (a) 'prefer stdlib over third-party' fires when writing code, (b) 'pin if you add it' fires when editing dependency manifests. Path-scoping the rule to manifests only (a tempting move because the sibling rule pin-dependencies.md is already scoped that way) would silently drop half (a) — the stdlib-first guidance would no longer fire when the agent is writing code. In the plugin manifest form, scope lives only in the rule file frontmatter; .tessl-plugin/plugin.json lists rule paths and carries no per-rule config. The plugin prescribes keeping mixed-scope rules universal (alwaysApply: true, no applyTo) so every prescription continues to fire in its native context. Baseline agents may either (1) default everything to alwaysApply: true and miss the deliberate decision, or (2) copy the pin-dependencies pattern and over-scope to manifests, which is the failure mode this rule guards against.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Rule file frontmatter declares alwaysApply: true",
"description": "The rules/stdlib-first.md file has a YAML frontmatter block with alwaysApply: true. The rule must stay universal because part (a) fires when writing code, not when editing manifests. Setting alwaysApply: false here (matching the pin-dependencies pattern) is the failure this scenario tests for — it would scope the rule to manifests only and silently drop the stdlib-first guidance when the agent is writing code",
"max_score": 24
},
{
"name": "Rule file frontmatter declares no scoping fields",
"description": "The rules/stdlib-first.md frontmatter does NOT contain applyTo, globs, or paths fields. Any scoping field narrows the rule's activation; with mixed-scope content, scoping to one context drops the other. The plugin prescribes omitting scoping fields when the rule's prescriptions span multiple contexts",
"max_score": 24
},
{
"name": "plugin.json rules array includes the new rule path",
"description": "The updated .tessl-plugin/plugin.json rules array includes the path rules/stdlib-first.md. Scores zero if the path is missing or wrong, or if the agent adds a steering map or a per-rule alwaysApply field to the manifest — that is the legacy tile.json model; plugin.json lists rule paths only",
"max_score": 14
},
{
"name": "Rule body covers the stdlib-first practice",
"description": "The rule body has at least one bullet covering the 'prefer stdlib over third-party' practice — this is the half that fires when writing code. Scores zero if the body only covers pinning and omits the stdlib-first guidance",
"max_score": 10
},
{
"name": "Rule body covers the dependency-pinning practice",
"description": "The rule body has at least one bullet covering the 'pin if you add it' practice — the manifest-level half. Scores zero if the body only covers stdlib-first and omits the pinning guidance",
"max_score": 10
},
{
"name": "Existing rules and manifest entries are preserved unchanged",
"description": "The three pre-existing rule paths (rules/commit-conventions.md, rules/spaces-not-tabs.md, rules/pin-dependencies.md) remain in the plugin.json rules array, and the inputs/rules/pin-dependencies.md file is not modified (its alwaysApply: false and applyTo must not be flipped or removed). Scores zero if any pre-existing path is dropped or the sibling rule file is altered",
"max_score": 18
}
]
}.tessl-plugin
evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10
scenario-11
scenario-12
scenario-13
scenario-14
scenario-15
scenario-16
scenario-17
scenario-18
rules
skills
adopt-fork-pr
eval-curation
install-reviewer