General-purpose coding policy for Baruch's AI agents
90
91%
Does it follow best practices?
Impact
90%
1.30xAverage score across 18 eval scenarios
Advisory
Suggest reviewing before use
{
"context": "Tests whether the agent, invoking the adopt-fork-pr skill, correctly diagnoses why a fork PR gets no policy review (the reviewer is fork-guarded / GitHub withholds secrets from fork-triggered runs) and brings the contributor's branch into the base repo so the reviewer fires — preserving the original commits (and their Author-Model trailer), opening a same-repo PR, linking the original, and leaving the original fork PR open. The branch-adoption move, commit preservation, and leave-original-open choice are tile-prescribed; a baseline agent typically tries to re-run the workflow, enable it on forks, or recreate the change from scratch.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Diagnoses why the fork PR is not reviewed",
"description": "Identifies that the reviewer does not run on pull requests opened from a fork — because the workflow is fork-guarded and/or GitHub does not expose repository secrets to fork-triggered runs — as the reason no review appeared. Does NOT misdiagnose it as a broken workflow, a missing secret to be added, or a transient failure to re-run",
"max_score": 20
},
{
"name": "Brings the branch into the base repo",
"description": "Pushes the contributor's branch to a branch on the base/origin repository (e.g. `git push origin HEAD:<branch>` after `gh pr checkout`), rather than asking the contributor to re-push, reconfiguring the workflow to run on forks, or using `pull_request_target` to hand secrets to fork runs",
"max_score": 20
},
{
"name": "Preserves the contributor's commits unchanged",
"description": "Carries the original commits across as-is — does NOT squash, rewrite, cherry-pick into a fresh commit, or hand-recreate the change. Preserving the commits is what keeps the contributor's authorship and any `Co-authored-by:` Author-Model trailer intact on the adopted PR",
"max_score": 18
},
{
"name": "Opens a same-repo PR from the adopted branch",
"description": "Opens a new pull request whose head branch lives in the base repo (`gh pr create` with the adopted branch as head), so the now-same-repo PR triggers the reviewer",
"max_score": 15
},
{
"name": "Leaves the original fork PR open",
"description": "Does NOT close, merge, or alter the original fork PR. Closing it is the contributor's decision; the skill leaves it open",
"max_score": 12
},
{
"name": "Links the adopted PR back to the original",
"description": "Comments on (or otherwise references) the original fork PR pointing at the adopted same-repo PR, so the contributor and the trail stay connected",
"max_score": 10
},
{
"name": "Does not fabricate an Author-Model declaration",
"description": "Relies on the carried-over commit trailer for the Author-Model signal rather than inventing a model the contributor never declared. If the original declared a model only in its PR body, adding that same declaration to the adopted PR is acceptable; making one up is not",
"max_score": 5
}
]
}.tessl-plugin
evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10
scenario-11
scenario-12
scenario-13
scenario-14
scenario-15
scenario-16
scenario-17
scenario-18
rules
skills
adopt-fork-pr
eval-curation
install-reviewer