CtrlK
BlogDocsLog inGet started
Tessl Logo

evilissimo/implementation-integrity-review

Reviews repositories, pull requests, diffs, and agent-generated code for reward hacking, fake completion, defensive theater, architectural bypasses, weakened guarantees, hidden fallbacks, and misleading abstractions.

98

1.09x
Quality

97%

Does it follow best practices?

Impact

100%

1.09x

Average score across 6 eval scenarios

SecuritybySnyk

Passed

No known issues

Overview
Quality
Evals
Security
Files

integrity-signals.mddocs/

Integrity Signals

Use this reference to build leads during an implementation-integrity review. A signal is not a finding by itself. Report it only after connecting the code to a contract violation, false confidence, hidden failure, architectural bypass, or misleading behavior.

Contract Mismatch Signals

  • Public API claims work is complete, but the implementation returns canned success, empty data, fixed IDs, or placeholder values.
  • A feature flag, provider, adapter, or mode is advertised but only one path exists, or alternate paths are unreachable.
  • Inputs are accepted but ignored, normalized away, or only handled for known fixture values.
  • State-changing operations log success without persisting, publishing, scheduling, or calling the required downstream dependency.
  • A fallback changes semantics while pretending the operation succeeded.

Weakened Guarantee Signals

  • Validation is removed, broadened, or moved behind permissive defaults.
  • Errors are swallowed, converted to success, or hidden behind generic status objects without caller-visible failure.
  • Authorization, idempotency, rate-limit, schema, or consistency checks are bypassed on new paths.
  • Retry, cache, async, or queue logic exists but does not preserve the original failure, ordering, timeout, or consistency contract.
  • Observability is reduced where failures matter: no structured log, metric, trace, audit event, or returned diagnostic remains.

Test And CI Signals

  • Tests assert weak properties such as "does not throw" where the contract needs concrete output, persistence, side effects, or error behavior.
  • Assertions, fixtures, or integration steps were removed without equivalent coverage elsewhere.
  • CI skips, marks, environment checks, or path filters exclude the changed code.
  • Tests exercise only mocked happy paths while production behavior depends on untested integrations.
  • Snapshot, golden-file, or broad regex assertions mask meaningful regressions.

Architecture Signals

  • New code bypasses established services, repositories, policies, capability checks, or transaction boundaries.
  • Data is duplicated or re-derived outside the authoritative layer.
  • Cross-cutting concerns such as auth, tenancy, audit, schema migration, or locking are reimplemented locally.
  • "Temporary" direct calls or imports create a second implementation of a domain rule.
  • A dependency direction is inverted to make a narrow task pass.

Abstraction Signals

  • A new interface, provider, plugin, registry, or strategy has one implementation and no immediate substitutability need.
  • Capability flags exist but are never exercised or validated.
  • Optional dependencies have no semantic fallback, only silent no-ops.
  • A compatibility layer catches import/runtime failures but does not preserve behavior, report degraded mode, or document unsupported features.
  • Resiliency wrappers such as retries, circuit breakers, and caches do not change the actual failure or latency behavior.

Evidence Standard

For each potential finding, verify:

  1. What behavior was promised by the request, docs, public API, tests, or architecture.
  2. Which exact code path violates or misrepresents that behavior.
  3. Which caller, operator, reviewer, or test receives false confidence.
  4. Whether an existing legitimate pattern explains the code.
  5. What targeted verification would prove or disprove the issue.

Drop the lead if it is only style, naming, local simplicity, or generic lint noise. Keep it if it changes externally visible behavior, hides failure, weakens guarantees, or makes the implementation appear more complete than it is.

docs

defensive-theater.md

integrity-signals.md

reward-hacking-patterns.md

scoring-model.md

SKILL.md

tile.json