CtrlK
BlogDocsLog inGet started
Tessl Logo

sahildmk/pr-comment-resolver

Review PR comments, address code issues in source files (not generated files), regenerate derived artifacts, run lint/format, commit, push, and reply to the comment thread confirming resolution.

92

1.21x
Quality

90%

Does it follow best practices?

Impact

97%

1.21x

Average score across 5 eval scenarios

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

Evaluation results

95%

3%

PR Comment Filtering Tool

Criteria
Without context
With context

Review comments endpoint

100%

100%

Authenticated user lookup

100%

100%

Exclude self-replied

100%

100%

Top-level only filtering

100%

100%

Agent commit timestamp

70%

100%

Since parameter usage

100%

100%

First-run handling

100%

100%

Stop when empty

50%

50%

Reply-user matching

100%

100%

Test coverage

100%

100%

100%

PR Review Comment Assessment Report

Criteria
Without context
With context

Three decision categories

100%

100%

Generated file recognition

100%

100%

OpenAPI generated file

100%

100%

Defer large refactor

100%

100%

Rationale provided

100%

100%

Defer/disagree reasoning

100%

100%

User confirmation plan

100%

100%

Plan shows all comments

100%

100%

Critical assessment

100%

100%

Source file targets only

100%

100%

100%

10%

Fix Review Feedback in a TypeScript API Project

Criteria
Without context
With context

Source schema edited

100%

100%

Generated types not hand-edited

100%

100%

Generation script executed

100%

100%

Lint verification

100%

100%

Minimal source changes

100%

100%

Handler updated consistently

100%

100%

Staged files listed

100%

100%

Commit message format

100%

100%

Co-authored-by line

0%

100%

Workflow ordering

100%

100%

100%

29%

GitHub PR Reply Automation Script

Criteria
Without context
With context

Replies endpoint

100%

100%

Top-level comment ID

100%

100%

Diagnose before fallback

100%

100%

Verify IDs on failure

100%

100%

Issue comment fallback last

50%

100%

Short addressed reply

100%

100%

Short defer reply

60%

100%

Disagree with reason

100%

100%

Commit message format

0%

100%

Co-authored-by attribution

0%

100%

91%

45%

PR Comment Resolver CLI Tool

Criteria
Without context
With context

Review comments API

100%

100%

Comment filtering stage

12%

100%

Assessment stage

60%

60%

User confirmation stage

100%

100%

Source-only editing

0%

100%

Regeneration stage

0%

50%

Verification stage

0%

100%

Correct workflow order

50%

100%

Reply via thread endpoint

100%

100%

One commit per comment

0%

87%

Commit format

50%

100%

Empty result handling

100%

100%

Evaluated
Agent
Claude
Model
Claude Sonnet 4.6

Table of Contents