markusdowne/detectability-contract
Creates boundary-point validation contracts, defines invariant-based success criteria, and sets up automated verification probes so reliability workflows trigger on objective evidence rather than intuition. Use when designing robust handoff, memory-persistence, or tool-call reliability workflows; when you need to verify handoffs work, check memory persistence, validate tool calls succeeded, or convert vague reliability goals into concrete, testable checks at each boundary point with explicit failure-class mapping (operational vs. critical); or when you want to test your workflow end-to-end, make sure it works, or verify your automation runs correctly using read-back probes and escalation triggers rather than agent confidence. Includes explicit untrusted-content/prompt-injection guardrails for third-party inputs.
96
Quality
90%
Does it follow best practices?
Impact
98%
1.25xAverage score across 9 eval scenarios
task.mdevals/scenario-4/
Payment API Integration Reliability Contract
Problem/Feature Description
A SaaS company integrates with a third-party payment processor. Their checkout workflow calls the payment API, receives a response, records the transaction, and emails a receipt. Two incidents in the past quarter involved responses that returned HTTP 200 but contained no transaction_id field — the processor was under load and returning partial responses. The application recorded these as successful payments, causing reconciliation failures discovered only days later.
The platform team is hardening the checkout pipeline and needs a formal detectability contract specifically for the external tool call boundary. The contract should define exactly what invariants to check after a payment API response is received, which verification probes to run, how to classify different failure modes, and when to escalate rather than silently retry.
Output Specification
Produce the following files:
contract.md— A contract document with a boundary table covering the external tool call (payment API call) boundary, with all required columns filled in.validate_response.py— A Python script that accepts a JSON file path as a command-line argument, reads a simulated API response from that file, and validates it against the invariants described in the contract. Required fields to check:transaction_id,status,amount,currency. The script should print a summary of which invariants passed or failed and exit with code 1 if any invariant fails.
Example input file format:
{"status": 200, "transaction_id": "txn_abc123", "amount": 99.99, "currency": "USD"}The script should be runnable with: python validate_response.py <path_to_response_file>