{
  "context": "The agent is given a system design draft for a distributed notification delivery system plus a project decision log, and asked to stress-test the design for gaps in failure/edge-case handling, then produce a gap analysis, concrete proposals, and parked open questions. This scenario tests whether the agent invents concrete awkward scenarios and traces them end-to-end to find gaps, mines existing project records before re-litigating settled decisions, surfaces contradictions as questions instead of assuming them away, triages aggressively so only genuine forks reach open-questions, organizes proposals by theme with dependency chains and safe-default/judgment-call tags, and parks unresolvable decisions as one-line 'topic: recommended answer' entries.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Concrete weird scenarios",
      "description": "stress-test-scenarios.md contains at least 4 distinct scenarios that are specific and concrete (e.g., same event_id arriving twice, adapter returns error mid-flight, flush job fires while dispatcher is also retrying) rather than generic 'what if it fails' statements",
      "max_score": 8
    },
    {
      "name": "Design silence identified",
      "description": "For each scenario, stress-test-scenarios.md identifies exactly where the design goes silent or contradicts itself, referencing specific components, config keys, or table columns from system-design.md",
      "max_score": 6
    },
    {
      "name": "End-to-end trace",
      "description": "Each scenario in stress-test-scenarios.md traces the event step-by-step through the design (dispatcher → adapter → ledger → Kafka offset), not just naming the failure mode in isolation",
      "max_score": 6
    },
    {
      "name": "Proposals are exact",
      "description": "gap-proposals.md proposals use exact concrete text (specific values, specific behavior) — e.g., 'retry up to 3 times with 30-second backoff' — rather than vague language like 'add a reasonable retry policy'",
      "max_score": 8
    },
    {
      "name": "Themed organization",
      "description": "gap-proposals.md groups proposals under a small number of named theme sections (e.g., 'Failure Handling', 'Duplicate Delivery', 'Flush Conflicts'), each containing its related proposals — individually titled proposals in one flat P1..Pn sequence do not count as themes",
      "max_score": 8
    },
    {
      "name": "Dependency chains noted",
      "description": "gap-proposals.md explicitly notes when one proposal builds on another (e.g., 'builds on Proposal 2' or equivalent phrasing) for at least one proposal that genuinely depends on a prior one",
      "max_score": 5
    },
    {
      "name": "Safe-default vs judgment-call tags",
      "description": "Each proposal in gap-proposals.md is tagged as either 'safe-default' or 'judgment-call' (or equivalent clear labeling distinguishing conventional/derivable choices from decisions with real trade-offs)",
      "max_score": 10
    },
    {
      "name": "Implied sub-decisions surfaced",
      "description": "gap-proposals.md contains at least one proposal that was not directly visible from a surface scan of the design — i.e., a sub-decision implied by filling a primary gap (e.g., 'if we add retries, what happens to the Kafka offset during the retry window?' reveals a ledger-state sub-decision)",
      "max_score": 8
    },
    {
      "name": "Open questions format",
      "description": "open-questions.md contains entries formatted as single-line 'topic: recommended answer' one-liners — each entry names the decision AND states a recommended resolution (not multi-paragraph explanations, and not bare questions without recommended answers)",
      "max_score": 10
    },
    {
      "name": "Triage discipline",
      "description": "open-questions.md contains at most 5 entries, each a genuine team-level fork (hard to reverse, real trade-off between real alternatives); decisions that are cheap to change or conventionally derivable are resolved as concrete proposals in gap-proposals.md instead of being escalated as questions — a laundry list of 10+ open questions fails this criterion",
      "max_score": 8
    },
    {
      "name": "Contradiction surfaced as question",
      "description": "The direct contradiction between the Happy Path (step 6: Kafka offset committed after ledger write) and Operational Notes (enable.auto.commit=true, offsets committed automatically every 5 seconds) is explicitly identified as two parts of the design disagreeing, and is escalated to open-questions.md for a team decision (with a recommended resolution) — not silently resolved by a proposal that picks one side without flagging the conflict",
      "max_score": 10
    },
    {
      "name": "Settled decisions respected",
      "description": "Outputs do not re-litigate or silently override decisions recorded in inputs/decisions.md: handling of Preferences Service failure stays consistent with 'dropped, not queued or retried' (e.g., proposals may specify how a drop is recorded in the ledger, but not introduce queuing/retry/fallback-cache behavior as if undecided), and no proposal introduces multi-provider failover; if the agent disagrees with a recorded decision, the disagreement appears as an explicit open question, not as a contradicting proposal",
      "max_score": 8
    },
    {
      "name": "No implementation deliverables",
      "description": "The output files contain analysis, proposals, and parked questions — not implementation deliverables (source code files, SQL migration files, config files, deployment scripts, or multi-line code/SQL blocks). Short inline illustrative fragments inside a proposal's prose (e.g., naming a unique constraint on (event_id, channel) or listing allowed status values) are acceptable and not penalized",
      "max_score": 5
    }
  ]
}