CtrlK
BlogDocsLog inGet started
Tessl Logo

evilissimo/implementation-integrity-review

Reviews repositories, pull requests, diffs, and agent-generated code for reward hacking, fake completion, defensive theater, architectural bypasses, weakened guarantees, hidden fallbacks, and misleading abstractions.

98

1.09x
Quality

97%

Does it follow best practices?

Impact

100%

1.09x

Average score across 6 eval scenarios

SecuritybySnyk

Passed

No known issues

Overview
Quality
Evals
Security
Files

SKILL.md

name:
implementation-integrity-review
description:
Reviews repositories, pull requests, diffs, and agent-generated code for implementation integrity failures: reward hacking, fake completion, defensive theater, test manipulation, architectural bypasses, weakened guarantees, and misleading abstractions. Use whenever the user asks for an integrity review, adversarial review, reward-hacking review, fake implementation review, shortcut audit, or verification that an implementation genuinely satisfies its stated contract.

Implementation Integrity Reviewer

Review code for deceptive implementation behavior, reward hacking, architectural shortcuts, defensive theater, fake completion, and false confidence. Prioritize evidence that the implementation does not honestly deliver the behavior it claims.

Required Reference Material

Before starting the review, read the relevant bundled references provided with this skill:

  1. Integrity Signals
  2. Reward Hacking Patterns
  3. Defensive Theater
  4. Scoring Model
  5. Integrity First Rule - use this as the final finding filter and severity tie-breaker.

Apply the integrity-first rule before finalizing findings: report issues that materially affect correctness, maintainability, architectural consistency, or operational transparency. Drop or demote observations that are only stylistic, generic lint noise, or superficial cleanup unless they directly weaken the implementation contract.

Review Objectives

Actively search for:

  • reward hacking, test-specific behavior, and weakened test guarantees
  • fake completion, hardcoded success, and placeholder production behavior
  • fake abstractions, fake resiliency, and non-substitutable providers or fallbacks
  • architectural boundary bypasses and scope dodging
  • weakened validation, swallowed failures, silent degradation, and observability gaps

Review Workflow

  1. Establish the claimed contract. Read the user request, PR description, tests, docs, public APIs, and any relevant architecture notes.
  2. Profile repository structure. Identify production code, tests, generated files, scripts, adapters, boundary modules, and existing architecture seams.
  3. Run deterministic scans when the project and available tools support them. Treat scan output as leads, not findings.
  4. Inspect tests and CI for weakened assertions, fixture special cases, skipped coverage, removed checks, and test-only branches.
  5. Compare implementation paths against the claimed contract. Follow data, side effects, error handling, persistence, external calls, retries, and observability paths end to end.
  6. Correlate suspicious signals. Connect each signal to behavior that misleads callers, reviewers, or tests.
  7. Produce evidence-backed findings only. If the code is legitimate, say that no integrity findings were identified.

Deterministic Scan Expectations

Use bundled scripts when helpful:

  • scripts/run-python-scans.sh for Python repositories
  • scripts/run-ts-scans.sh for TypeScript or JavaScript repositories
  • scripts/collect-signals.py <path> for lightweight pattern-based leads

Evaluation scenarios are indexed in evals/README.md.

The scripts skip missing tools and continue running available checks. Do not present missing optional tools as implementation findings.

Recommended tools: Ruff, Semgrep, basedpyright, vulture, and lint-imports for Python; ESLint, dependency-cruiser, knip, and ts-prune for TypeScript or JavaScript.

False Positive Control

Do not punish simple code for being simple. A finding needs evidence that the implementation creates false confidence or violates a stated contract.

Treat adapters, optional dependencies, defensive error handling, and narrow scope as legitimate when they preserve semantics, report failures clearly, and are covered by tests.

Output Requirements

Lead with findings ordered by severity. In file-oriented workspaces or eval scenarios, write the review to IMPLEMENTATION_INTEGRITY_REVIEW.md; otherwise answer directly in the conversation. Every finding must include:

  • category
  • severity
  • confidence
  • affected file(s)
  • evidence with exact code references
  • rationale explaining the integrity failure
  • recommended verification or remediation

If there are no integrity findings, say that directly and list any residual review limits such as unavailable tools or uninspected runtime behavior.

SKILL.md

tile.json