CtrlK
BlogDocsLog inGet started
Tessl Logo

three-way-judge

Run an explicitly requested three-option workflow with three independent executors, one independent paired critic per option, bounded revision loops, and a blind evidence-backed judge. Use only when the user invokes $three-way-judge or explicitly requests this exact panel; this v1 skill does not run the nine-review deep mode.

70

Quality

85%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Three-Way Judge

Produce three genuinely independent solutions, evidence-gate each lane, and select a winner without grade inflation or forced consensus.

Required protocol

Before dispatch, read:

  • references/panel-protocol.md for waves, lane isolation, blind judging, and terminal outcomes.
  • references/rubric.md for the critic and judge acceptance gate.
  • references/judge-rubric.md for blind rescoring and comparative ranking.
  • references/security-screen.md for the bounded security pass.
  • references/evidence-protocol.md before creating requirement gates, evidence manifests, or verification records.
  • references/scorecard-schema.md and references/panel-schema.md before persisting results or run manifests.

Non-negotiable rules

  1. Freeze one task packet and give the identical packet to all three executors and critics.
  2. Run exactly three independent minimal-context execution lanes. When history controls are available, start each role without inherited conversation history and pass only its frozen task packet and necessary artifacts. Executors must not see or revise from another lane's work.
  3. Assign one read-only independent critic to each lane. Each critic uses the same rubric and may conduct at most three total reviews.
  4. Give every frozen must-have specification a binary requirement gate. A lane cannot PASS unless every MUST gate is independently VERIFIED through linked evidence.
  5. Fingerprint the shared task packet, every command-consumed file, and each lane's artifact set, evidence manifest, and evidence result. Any content change invalidates dependent scores.
  6. Only a lane with critic status PASS advances as an eligible candidate. Never average critic scores.
  7. Generate and fingerprint a neutral blind packet containing anonymous candidate IDs, task packet, artifacts, and primary evidence only—no authorship, critic grades, defects, or revision history. The blind judge may read only that packet and its linked files.
  8. After blind scoring, reveal critic evidence and revision ledgers for reconciliation. Every numerical score change must name its rubric criterion, exact delta, independently verified evidence IDs, and rationale.
  9. A judge PASS requires at least 85.00/100, all category floors, no hard failures, no critical unknowns, verified evidence links, and a valid v2.1 scorecard.
  10. The judge may return WINNER, SINGLE_SURVIVOR, NO_WINNER, or BLOCKED. Never force a winner.
  11. The judge is read-only and must not synthesize an unreviewed fourth option. Any requested synthesis becomes a new execution artifact requiring a new critic gate.
  12. This skill does not run three critics per candidate. Do not silently expand it to nine reviews.
  13. Review approval does not expand authorization to publish, deploy, merge, delete, purchase, message, or otherwise mutate external state.

Runtime behavior

  • Use rolling dispatch that respects available concurrency: start independent executors when slots allow, fingerprint each returned artifact, and start its paired critic in the next available slot without waiting for the other executors. Start the blind judge only after every lane is terminal. On a four-slot system, the orchestrator occupies one slot and up to three roles run concurrently.
  • Give the three executors distinct approach mandates appropriate to the task: conservative/direct, optimized/simplified, and differentiated/ambitious. These are diversity constraints, not permission to change the task.
  • Keep each executor paired with its critic through revisions. A re-review verifies the complete artifact and all prior defects.
  • If independent contexts are unavailable, return BLOCKED: INDEPENDENCE_UNAVAILABLE; never simulate a panel in one context.
  • If a lane fails after review three, stop that lane. The judge evaluates only critic-passing lanes while still reporting all three outcomes.
  • Before any critic or judge awards PASS, require one bounded improvement sweep recorded as NO_MATERIAL_IMPROVEMENT_FOUND or IMPROVEMENTS_IDENTIFIED with linked evidence.
  • Apply the stalled-agent recovery policy per lane and to the judge. A critic round exists only when a complete scorecard returns; stalled attempts do not consume a round.

Verify each lane's approved evidence without a shell and validate every v2.1 critic and judge scorecard:

node <skill-directory>/scripts/verify-evidence.mjs --actor <CRITIC-or-JUDGE> --allow-command <executable> --report <evidence-result.json> <evidence-manifest.json>
node <skill-directory>/scripts/validate-scorecard.mjs <scorecard.json>

Then validate the panel decision:

node <skill-directory>/scripts/validate-panel.mjs <panel-result.json>

Before blind judging, generate the neutral packet from the lane evidence records and fingerprint the result:

node <skill-directory>/scripts/build-blind-packet.mjs --output judge/blind-packet.json <blind-source.json>

After judging, the orchestrator must independently rerun every lane's evidence and validate the complete linked run:

node <skill-directory>/scripts/verify-review-run.mjs --allow-command <executable> --report <verified-report.json> <panel-run.json>

Do not report a winner or survivor unless root verification succeeds. V1 and v2.0 records remain validator-compatible for archival inspection, but cannot complete a new run.

Return the machine-generated three-lane statuses and scores, requirement-gate counts, review-round counts, blind and reconciled judge scores, selected candidate or no-winner result, comparative rationale, security statuses, unresolved defects, evidence locations, hashes, and root-verification result.

Repository
stevenknowswhy/three-critic-judge-skill
Last updated
First committed

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.