CtrlK
BlogDocsLog inGet started
Tessl Logo

sahildmk/pr-comment-resolver

Review PR comments, address code issues in source files (not generated files), regenerate derived artifacts, run lint/format, commit, push, and reply to the comment thread confirming resolution.

93

1.19x
Quality

89%

Does it follow best practices?

Impact

99%

1.19x

Average score across 5 eval scenarios

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

Evaluation results

95%

5%

PR Comment Filtering Tool

PR comment filtering logic

Criteria
Without context
With context

Review comments endpoint

100%

100%

Authenticated user lookup

100%

100%

Exclude self-replied

100%

100%

Top-level only filtering

100%

100%

Agent commit timestamp

60%

100%

Since parameter usage

100%

100%

First-run handling

100%

100%

Stop when empty

40%

50%

Reply-user matching

100%

100%

Test coverage

100%

100%

100%

2%

PR Review Comment Assessment Report

Comment assessment and user confirmation

Criteria
Without context
With context

Three decision categories

100%

100%

Generated file recognition

100%

100%

OpenAPI generated file

80%

100%

Defer large refactor

100%

100%

Rationale provided

100%

100%

Defer/disagree reasoning

100%

100%

User confirmation plan

100%

100%

Plan shows all comments

100%

100%

Critical assessment

100%

100%

Source file targets only

100%

100%

100%

10%

Fix Review Feedback in a TypeScript API Project

Source-only editing and artifact regeneration

Criteria
Without context
With context

Source schema edited

100%

100%

Generated types not hand-edited

100%

100%

Generation script executed

100%

100%

Lint verification

100%

100%

Minimal source changes

100%

100%

Handler updated consistently

100%

100%

Staged files listed

100%

100%

Commit message format

100%

100%

Co-authored-by line

0%

100%

Workflow ordering

100%

100%

100%

15%

GitHub PR Reply Automation Script

Commit format and reply workflow

Criteria
Without context
With context

Replies endpoint

100%

100%

Top-level comment ID

100%

100%

Diagnose before fallback

100%

100%

Verify IDs on failure

100%

100%

Issue comment fallback last

100%

100%

Short addressed reply

80%

100%

Short defer reply

70%

100%

Disagree with reason

100%

100%

Commit message format

100%

100%

Co-authored-by attribution

0%

100%

100%

48%

PR Comment Resolver CLI Tool

End-to-end PR comment resolver workflow

Criteria
Without context
With context

Review comments API

100%

100%

Comment filtering stage

62%

100%

Assessment stage

90%

100%

User confirmation stage

100%

100%

Source-only editing

0%

100%

Regeneration stage

0%

100%

Verification stage

0%

100%

Correct workflow order

40%

100%

Reply via thread endpoint

100%

100%

One commit per comment

0%

100%

Commit format

50%

100%

Empty result handling

100%

100%

Evaluated
Agent
Claude Code
Model
Claude Sonnet 4.6

Table of Contents