markusdowne/detectability-contract
Creates boundary-point validation contracts, defines invariant-based success criteria, and sets up automated verification probes so reliability workflows trigger on objective evidence rather than intuition. Use when designing robust handoff, memory-persistence, or tool-call reliability workflows; when you need to verify handoffs work, check memory persistence, validate tool calls succeeded, or convert vague reliability goals into concrete, testable checks at each boundary point with explicit failure-class mapping (operational vs. critical); or when you want to test your workflow end-to-end, make sure it works, or verify your automation runs correctly using read-back probes and escalation triggers rather than agent confidence. Includes explicit untrusted-content/prompt-injection guardrails for third-party inputs.
96
Quality
90%
Does it follow best practices?
Impact
98%
1.25xAverage score across 9 eval scenarios
task.mdevals/scenario-2/
Data Pipeline Reliability Contract
Problem/Feature Description
A data engineering team runs a nightly ETL pipeline with three stages: (1) raw extraction from a source database, (2) transformation and enrichment, and (3) loading into the analytics warehouse. Twice this month the warehouse contained stale or zero-row tables, yet the pipeline's status dashboard showed green across all stages. Engineers traced the cause to silent failures during the file-based handoff between stages: the extractor wrote a malformed JSON file and the transformer silently skipped it, producing empty output.
The team needs a formal detectability contract for this pipeline so that any silent failure at a file handoff boundary triggers an explicit error, not a quiet skip. The contract should be implementable as an automated check that runs after each stage completes and before the next stage begins.
Output Specification
Produce the following files:
contract.md— A contract document containing a boundary analysis table that covers the handoff points in this pipeline, with all required columns filled in for each boundary.verify.py— A Python script implementing the invariant checks described in the contract for the file handoff boundaries. The script should accept a file path as a command-line argument and exit with a non-zero code if any invariant is violated, printing the specific failure.
The Python script should be runnable with: python verify.py <path_to_artifact>
Do not include placeholder comments — implement real checks that a team could drop into a CI pipeline.