testland/steady-state-hypothesis-validator
Validates a chaos experiment's steady-state hypothesis before execution: checks that each probe metric is measurable and observable, that a recent baseline exists, that tolerances are numerically meaningful and SLI-backed, that the measurement window is defined, and that the chosen metrics would actually move under the target failure mode. Use when a chaos experiment has been authored (via chaos-experiment-author) and the team needs a pre-flight verdict before running the drill in any environment.
70
88%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
View guide
Low
Low-risk findings worth noting
chaostoolkit-tolerance.mdreferences/
Chaos Toolkit steady-state-hypothesis block and tolerance forms
Reference for steady-state-hypothesis-validator. The five pre-flight checks in
SKILL.md validate a hypothesis expressed in this schema; this file is the schema
and tolerance detail they assume.
The steady-state-hypothesis object
Per chaostoolkit.org/reference/api/experiment/, the
steady-state-hypothesis object requires:
title(string): human-readable rationale for the hypothesis.probes(array): one or more probe objects, each with:type:"probe"name: identifier stringprovider: execution specification (HTTP, process, or Python)tolerance: the gate value; if the probe's return value does not satisfy the tolerance, the experiment bails before running the method.
Tolerance forms supported
Per chaostoolkit.org/reference/api/experiment/:
| Tolerance form | Syntax example | Evaluation |
|---|---|---|
| Scalar equality | "tolerance": 200 | probe return == 200 |
| Boolean equality | "tolerance": true | probe return == true |
| String equality | "tolerance": "OK" | probe return == "OK" |
| Inclusive range | "tolerance": [95, 100] | 95 <= value <= 100 |
| Membership | "tolerance": [200, 201, 204] | value in list |
| Regex | "tolerance": {"type": "regex", "pattern": "^healthy$"} | regex match |
| JSONPath | "tolerance": {"type": "jsonpath", "path": "$.status", "expect": "up"} | JSONPath extract + compare |
| Range object | "tolerance": {"type": "range", "range": [95.0, 100.0]} | numeric bounds |
Execution flow
Per chaostoolkit.org/reference/concepts/: probes run once before the method (baseline check) and once after (deviation check). A probe that fails before the method means the system is already outside its acceptable state; the experiment must not run. A probe that fails after the method means the chaos activity caused the system to leave its steady state.