testland/chaos-drill-protocol
Run protocol and run workflow for a chaos experiment that has already been designed: the four pre-flight gates (non-production target, measured healthy baseline, live observability, a rollback that has actually been exercised), how to pick a conservative blast-radius bound, the sampling cadence and abort criteria fixed in writing before injection, the per-runner inject and abort commands (Chaos Mesh / Litmus / Gremlin / Toxiproxy), the refuse-to-start rules (no blast-radius bound, production context, degraded baseline, offline observability, unexercised rollback), and the recovery-validation step with its tolerance and timeout. Owns execution safety only, not experiment design: the steady-state hypothesis, the fault to inject, and the experiment file come from chaos-experiment-author. Use when an experiment definition exists and a fault is about to be injected into a running system, and the go/no-go gates, abort thresholds, and recovery check still need to be agreed and written down before the fault starts.
72
91%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
View guide
Medium
Suggest reviewing before use
golden-signals.mdreferences/
The four golden signals
Reference for chaos-drill-protocol Gate 3 (live observability). The signals a
drill confirms and aborts on are the ones its abort criteria name. A useful
default set is the four golden signals from Google's SRE monitoring chapter
(sre.google/sre-book/monitoring-distributed-systems).
| Signal | Definition (sre.google/sre-book/monitoring-distributed-systems) |
|---|---|
| Latency | "The time it takes to service a request. It's important to distinguish between the latency of successful requests and the latency of failed requests." |
| Traffic | "A measure of how much demand is being placed on your system, measured in a high-level system-specific metric." |
| Errors | "The rate of requests that fail, either explicitly (e.g., HTTP 500s), implicitly (for example, an HTTP 200 success response, but coupled with the wrong content), or by policy." |
| Saturation | "How 'full' your service is. A measure of your system fraction, emphasizing the resources that are most constrained." |
Gate 3 passes only when fresh samples for every abort-criterion signal have arrived within the last sampling interval. Stale dashboards fail the gate as hard as missing ones: without live signals the drill has no abort path that fires on evidence, which makes it a fault injection with no stopping rule rather than a riskier drill.