CtrlK
BlogDocsLog inGet started
Tessl Logo

jbaruch/coding-policy

General-purpose coding policy for Baruch's AI agents

90

1.30x
Quality

91%

Does it follow best practices?

Impact

90%

1.30x

Average score across 18 eval scenarios

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-11/

{
  "context": "Tests whether the agent, invoking the adopt-fork-pr skill, correctly diagnoses why a fork PR gets no policy review (the reviewer is fork-guarded / GitHub withholds secrets from fork-triggered runs) and brings the contributor's branch into the base repo so the reviewer fires — preserving the original commits (and their Author-Model trailer), opening a same-repo PR, linking the original, and leaving the original fork PR open. The branch-adoption move, commit preservation, and leave-original-open choice are tile-prescribed; a baseline agent typically tries to re-run the workflow, enable it on forks, or recreate the change from scratch.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Diagnoses why the fork PR is not reviewed",
      "description": "Identifies that the reviewer does not run on pull requests opened from a fork — because the workflow is fork-guarded and/or GitHub does not expose repository secrets to fork-triggered runs — as the reason no review appeared. Does NOT misdiagnose it as a broken workflow, a missing secret to be added, or a transient failure to re-run",
      "max_score": 20
    },
    {
      "name": "Brings the branch into the base repo",
      "description": "Pushes the contributor's branch to a branch on the base/origin repository (e.g. `git push origin HEAD:<branch>` after `gh pr checkout`), rather than asking the contributor to re-push, reconfiguring the workflow to run on forks, or using `pull_request_target` to hand secrets to fork runs",
      "max_score": 20
    },
    {
      "name": "Preserves the contributor's commits unchanged",
      "description": "Carries the original commits across as-is — does NOT squash, rewrite, cherry-pick into a fresh commit, or hand-recreate the change. Preserving the commits is what keeps the contributor's authorship and any `Co-authored-by:` Author-Model trailer intact on the adopted PR",
      "max_score": 18
    },
    {
      "name": "Opens a same-repo PR from the adopted branch",
      "description": "Opens a new pull request whose head branch lives in the base repo (`gh pr create` with the adopted branch as head), so the now-same-repo PR triggers the reviewer",
      "max_score": 15
    },
    {
      "name": "Leaves the original fork PR open",
      "description": "Does NOT close, merge, or alter the original fork PR. Closing it is the contributor's decision; the skill leaves it open",
      "max_score": 12
    },
    {
      "name": "Links the adopted PR back to the original",
      "description": "Comments on (or otherwise references) the original fork PR pointing at the adopted same-repo PR, so the contributor and the trail stay connected",
      "max_score": 10
    },
    {
      "name": "Does not fabricate an Author-Model declaration",
      "description": "Relies on the carried-over commit trailer for the Author-Model signal rather than inventing a model the contributor never declared. If the original declared a model only in its PR body, adding that same declaration to the adopted PR is acceptable; making one up is not",
      "max_score": 5
    }
  ]
}

README.md

tile.json