CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

lightrun-runtime-aware-pr-review

Use when reviewing a pull request with runtime or production evidence — for example to review a PR with runtime verification, gather production evidence, or simulate a patch on live samples. Reviews a pull request by diffing against the PR merge base, checking Runtime History before collecting live samples for remaining gaps, and simulating the patch on captured production inputs.

Quality

89%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

SKILL.md
Quality
Evals
Security

Goal

Provide a single PR review backbone that uses runtime production evidence when possible and falls back to scenario-driven review when runtime verification is incomplete.

Scope

  • In scope: PR merge-base diffing, deployed-revision resolution for instrumentation, runtime verifiability checks, Runtime History evaluation, live sampling, runtime patch simulation with PR-base vs PR-head attribution, and review findings.
  • Out of scope: code changes, rollout execution, repository-specific automation beyond what is documented in the phase references.

How To Use This Skill

Read and follow these references in order. Each phase produces a specific artifact; confirm that artifact exists and meets the phase's exit criteria before advancing to the next phase.

  1. Intro — agent role, the process steps, and stage-vs-automation output separation
  2. Constraints — the hard rules every phase must obey
  3. Code management — GitHub or Bitbucket access, PR metadata, merge base, and diffs
  4. Phase 0 - Get deployed commit — PR base/head SHAs, PR diff, and optional deployed revision
  5. Phase 1 - Feasibility check — a verifiable / not-verifiable determination
  6. Phase 2 - Verification areas — PR diff areas mapped to deployed equivalents
  7. Phase 3 - Runtime History — evaluate existing Runtime History for mapped areas before live sampling
  8. Phase 4 - Sampling plan — snapshot entries for areas not fully covered
  9. Phase 5 - Execute snapshots — captured production samples at deployed locations only
  10. Phase 6 - Patch simulation — PR-base vs PR-head behavior, runtime assessment, and scenario-driven findings
  11. Output format — the final automation-response transcript

Checkpoints

Before moving from one phase to the next, confirm the current phase's exit criteria are met:

  • After Phase 0: pr_base_sha, pr_head_sha, and the PR diff (pr_base_sha → pr_head_sha) are in hand; deployed_sha is recorded when resolved.
  • After Phase 1: feasibility is determined; if not verifiable, the run stops here with the Phase 1 message.
  • After Phase 2: every PR verification area has a deployed mapping and status (uncovered or not runtime-verifiable).
  • After Phase 3: record history availability and evidence provenance; classify mapped areas as covered, base-covered, head-covered, or uncovered per Phase 3. Skip live planning and capture when all are covered.
  • After Phase 4: snapshot entries exist for the highest-value base-covered, head-covered, or uncovered verification areas (up to 3).
  • After Phase 5: every verification location has at least one hit, or the run stopped with a Sampling Request because one or more locations still had zero hits (resume required before Phase 6).
  • After Phase 6: the runtime assessment is justified with PR-base vs PR-head behavior on valid samples, and the full Output Format response is emitted.

Do not advance a phase until its exit criteria are satisfied; loop back to the prior phase when evidence is incomplete.

Working Rules

  • Detect GitHub or Bitbucket from the PR URL and use that host via Code management.
  • Use the PR diff (pr_base_sha → pr_head_sha) for all verification areas, attribution, findings, runtime assessment, and merge recommendation.
  • Use deployed_sha only to locate safe runtime instrumentation and describe true deployed production behavior.
  • Never use deployed_sha → pr_head_sha as the PR review diff.
  • Never query, instrument, or snapshot PR-only or unmapped coordinates.
  • Follow MCP tool discovery for history reads. Check suitable history first; create live snapshots only for mapped areas marked base-covered, head-covered, or uncovered. History reads do not require active agents or live-source preflight.
  • Keep phase messages and the final automation output separate per the intro reference.
  • For async live snapshots: partial hits do not complete Phase 5 — if any verification location still has zero hits after the bounded in-session poll, emit the Sampling Request and stop instead of proceeding to Phase 6.
  • Follow every constraint in Constraints.

Deliverable

At the end of a completed run, emit the full automation response described in Output format. Keep intermediate phase communication in chat using only the content defined for that phase.

Repository

lightrun-platform/lightrun-ai

Path

skills/lightrun-runtime-aware-pr-review/SKILL.md

Last updated
First committed

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.