When validating a coralogix terraform-provider bug fix end-to-end against a real Coralogix env. Triggers on "test the fix locally", "reproduce the bug", "validate before merge".
78
66%
Does it follow best practices?
Impact
99%
1.98xAverage score across 3 eval scenarios
Passed
No known issues
Optimize this skill with Tessl
npx tessl skill review --optimize ./.claude/skills/manual-validation-harness/SKILL.mdTrigger: user wants to exercise a provider bug fix against a real env, post-build pre-merge.
Procedure:
make install from the PR branch (builds + installs to ~/.terraform.d/plugins/locally/debug/coralogix/1.5/<arch>).mkdir /tmp/<ticket>-test && cd $_.terraformrc with dev_overrides for coralogix/coralogix pointing at the install dir above. export TF_CLI_CONFIG_FILE=$PWD/terraformrc../steps/, cp steps/stepN.tf main.tf between runs. (Terraform reads every .tf in cwd → loose step files cause duplicate-resource errors.)terraform apply -auto-approve, verify in UI, then terraform plan → expect No changes. (idempotency catches perpetual-diff regressions).terraform destroy -auto-approve && rm -rf /tmp/<ticket>-test.Gotchas:
.tf files via heredoc in chat — paste indentation breaks them. Use an editor or have Claude Write them.Why: unit + acceptance tests catch regressions; manual validation gives the reviewer end-to-end confidence and exposes UI-layer behaviour the test suite can't observe.
f197371
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.