Lesson 4: Review on every pull request

Everything so far printed to your terminal, which means it only happened when you remembered to ask. This lesson puts the review on the pull request, and then does the thing the course is named for: a second round that knows what the first one said.

You'll need a GitHub repository you're happy to experiment in, since this publishes real reviews on real pull requests.

What you'll build

• A caller workflow that reviews pull requests in your repository.
• One full round trip: findings, a fix, a reply pushing back, and a second review that settles both.
• An informed position on whether the review should block a merge, and what has to be true first.

The CLI prints, the Action publishes

tessl code review posts nothing to GitHub, by design. Putting reviews on pull requests is the job of the tesslio/code-review-action GitHub Action.

The split is clean, and knowing which side owns what saves a lot of debugging.

Your repository owns the caller workflow: when reviews run, what permissions the job holds, which runner, the timeout, the token secret, and whether findings block a merge.

The Action owns running a review: resolving the pull request, checking out the exact head, setting up the CLI, publishing the review, refusing to publish against a head that has moved, reconciling retries so you don't get two reviews, reporting its own check run, and uploading the result artifact.

Two consequences follow directly. Don't add a checkout step to the caller job, because the Action does its own, deliberately, with credentials disabled. And don't add a second thing that publishes; retry reconciliation and duplicate prevention live in one place, and a second publisher gives the repository two paths that can disagree.

Your profile does not come with you

The .tessl-code-review.yml you wrote last lesson routes local reviews. In CI, the Action takes a trusted named profile and an optional list of lenses as an input, and it does not load review policy from the pull request.

That reads like a gap until you look at it from the other direction. A pull request can change any file in the repository, including, if it were allowed to, the file deciding how that same pull request gets reviewed. A change that can rewrite its own reviewer isn't being reviewed.

So the policy CI runs comes from the caller workflow, which lives on the default branch and changes only when someone merges a change to it. Same idea as the actor check and the pinned Action revision below: what's trusted is what you merged, not what's in the branch under review.

Step-by-step

1. Add the token

Create an API key and store it as a repository secret named TESSL_TOKEN:

tessl api-key create --workspace <your-workspace> --name "code-review" --role member

Add it under Settings → Secrets and variables → Actions. The workflow passes it to the Action through the tessl-token input. It doesn't belong in the workflow file, an environment file, a command argument, or a comment.

Nobody can create that secret for you, so this step is yours whichever route you take next.

2. Add the workflow

Ask an agent, or write it yourself.

The setup plugin ships inside the CLI, so an agent can do this with nothing to install. Ask it:

Set up Tessl Code Review for this repository. Start in advisory mode and review a pull request when it becomes ready.

It reads your existing workflows first, asks the two questions below, and writes the file only once you approve it.

By hand, create .github/workflows/tessl-code-review.yml:

name: Tessl Code Review

on:
  pull_request:
    types: [opened, reopened, ready_for_review]
  issue_comment:
    types: [created]

permissions:
  contents: read
  checks: write
  issues: write
  pull-requests: write

concurrency:
  group: tessl-code-review-${{ github.event.pull_request.number || github.event.issue.number }}
  cancel-in-progress: false

jobs:
  review:
    if: >-
      (github.event_name == 'pull_request' && github.event.pull_request.draft == false) ||
      (github.event_name == 'issue_comment' &&
        github.event.issue.pull_request != null &&
        github.event.issue.state == 'open' &&
        contains(github.event.comment.body, '@tessl-code-review') &&
        contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association))
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
      - uses: tesslio/code-review-action@<full-commit-sha>
        with:
          tessl-token: ${{ secrets.TESSL_TOKEN }}
          profile: standard
          mode: advisory
          pr-number: ${{ github.event.issue.number }}

Two decisions are baked into that file.

When reviews run. This workflow reviews once when a pull request becomes ready, then again whenever someone asks. Pushing commits doesn't spend a review on its own. Add synchronize to the pull_request types if you want every pushed head reviewed, or drop the pull_request trigger entirely for a manual trial. Start here: it avoids reviewing draft work and lets the author decide when another round is worth it.

Whether findings block. mode: advisory publishes a comment review and never fails on a finding. Gate mode comes later in this lesson, and later in your rollout.

3. Don't skip these four

Pin to a full commit SHA. A tag is a moving reference: it can change after you've read the code it pointed at. The pin is the only thing bounding what runs with your token. Take the SHA from the release notes.

The author_association check is yours to make. A comment-driven run holds write permissions and your TESSL_TOKEN, and anyone who can comment on the pull request can start one. The Action authenticates nothing about the commenter. Without a condition like this, an outside commenter can spend your credits at will. Tighten the list to suit the repository.

@tessl-code-review is text, not an account. Recognizing it is your workflow's job, which is why it's in the if: condition. It also only works on Conversation-tab comments, and only once this file is on the default branch.

Forks are out of scope. Pull requests from forked repositories are rejected before a review runs. Don't reach for pull_request_target to get around it. It runs a privileged token against untrusted code, which is the exact boundary this design exists to hold.

The four permissions each buy something specific:

Permission	Why
contents: read	Resolve and check out the reviewed head
checks: write	Report the result as a check run
pull-requests: write	Publish the review itself
issues: write	Publish and clear failure notices

Drop checks: write and the review still completes. You just get no check run, and a warning naming what's missing. Nothing depends on it except branch protection, which is why gate mode needs it.

4. Open the pull request

Push the branch and open a pull request. Marking it ready starts the first review.

Three things land when it finishes:

• A review on the pull request, with a short judgment, a severity summary, and findings anchored to the lines they're about.
• A check run named Tessl Code Review, reported against the head that was actually reviewed.
• A result artifact on the workflow run, carrying the outcome, findings, configuration, duration, and publication receipt.

The outcome will be Changes requested (3), give or take what the model surfaces on the day.

5. Respond two different ways

The loop only means something if you do both of these.

Fix one. Move the null check above the dereference, commit, push.

Push back on another. Pick a finding you don't intend to act on and reply in its thread saying why. For this branch, the log line is a reasonable candidate: say it's behind a debug flag that never runs in production, if that's the story you want to tell.

Pushing commits doesn't start a review on this cadence, which is the point of it. When you're ready, comment:

@tessl-code-review

6. Read what the second round did

This is the part worth slowing down for. The review reads the new head and the conversation, and settles each earlier finding into one of three states:

• addressed: the code changed and the concern is gone
• explained: you replied with a reason it accepted
• declined: the concern stands and you chose not to act

Your fix should come back addressed. Your reply should come back explained or declined, depending on whether it accepted the reasoning. The finding you left alone should still be open, and it should not be restated from scratch as though nobody had ever seen it.

That's the whole difference between a reviewer and a linter with opinions. A tool that re-reads the diff cold will raise the same three points every round until you mute it.

Two things you'll hit eventually. A superseded run: push while a review is running and the Action publishes nothing for the head it was reading, because that head isn't the pull request's any more. The job fails, so an unpublished review can't be mistaken for a completed one, and the check goes neutral. Nothing needs fixing. And declined isn't quite pushback: the three categories give you one way to say "I'm not acting on this", and it concedes the concern was valid. There's no settled way to say a finding was simply wrong. If you disagree on the substance, say so plainly in the thread rather than expecting the category to carry it.

Advisory first, gate when it's earned

Advisory is the default, and it's the right place to sit for a while. The Action publishes a comment review, and no finding fails the check. A technical failure to run or publish still fails the job, and that distinction is deliberate.

Run it on real pull requests before you consider gating, and judge four things:

• Are the required changes accurate, and can someone act on them without a translation step?
• Are the optional suggestions worth their space, or are they noise?
• Do the lenses cover the risks that actually matter in this repository?
• Do later rounds settle earlier findings correctly?

That last one is the one people skip, and it's the one a gate depends on. If round two can't tell what you fixed, a gate turns every pull request into an argument.

Don't require the check while you're still advisory. Advisory never concludes failure, so a required advisory check can't gate anything. What it tells you is whether the review reached a verdict, not whether your code is fine.

When you're ready, four steps in order:

1. In Settings → Actions → General, allow GitHub Actions to create and approve pull requests. Approval doesn't work without it.
2. In Settings → Rules or branch protection, check nothing restricts who may review pull requests on the protected branch. If reviews are limited to a named set of people, GitHub refuses the Action's review event.
3. Change the workflow input to mode: gate.
4. Run it on a change that should pass and one that should be blocked, then add Tessl Code Review as a required check.

Require the check named exactly Tessl Code Review, not the caller workflow's job. Requiring the job only enforces on pull_request runs; a comment-driven run is associated with the default branch, so the job's status doesn't reliably reach the pull request and the required check may never arrive. The Action resolves the reviewed head itself, so its own check lands on that head whatever the trigger was.

Gate mode also fails closed. If the review returns no approval verdict, the check fails. An unjudged head doesn't pass through on the grounds that nothing said no.

One operational consequence to settle before you turn it on: on any cadence other than every-commit, a blocked pull request doesn't unblock itself. Pushing a fix starts nothing. Someone has to ask for the round. Make that part of how the team merges, or you'll have people waiting on a review that was never requested.

Verify

Six checks that the whole thing works, not just the parts you watched:

• Marking a pull request ready starts a review, and a check run named Tessl Code Review appears against the reviewed head. No check run usually means checks: write is missing from the workflow permissions.
• The published review carries findings anchored to the changed lines, and a count that matches the outcome.
• Your own lens is in there: the error-shape finding appears alongside the correctness and privacy ones. If it doesn't, check the lens list the Action actually ran, since the repository profile doesn't travel to CI.
• Mentioning @tessl-code-review starts a second round. If nothing happens, confirm the workflow is on the default branch, the comment is on the Conversation tab rather than inline on the diff, and the commenter matches your actor check. Editing an existing comment to add the mention won't do it.
• The second round reports what became of the first round's findings (addressed, explained or declined) rather than restating them.
• The workflow run carries a result artifact with the outcome, findings, configuration and duration.

If a run reports it was superseded and the job failed, nothing is wrong. A newer commit replaced the head being reviewed, so nothing was published for it. Only run it again if your triggers don't cover the push that replaced it.

What you keep

The workflow is merged, so pull requests in that repository get reviewed whether or not anyone remembers to ask. The lens you wrote in lesson 2 is part of that review, which takes one recurring check off the pile your reviewers work through by hand and leaves them the parts that need judgment.

You've also seen the loop close: a finding, a fix, a reply, and a second round that could tell the difference between them. That's the behavior a gate would depend on, and having watched it work on your own pull request is a better basis for turning one on than a vendor's claim about it.

Advisory is a fine place to stay for now. Gate when the reviews have earned it.