CtrlK
BlogDocsLog inGet started
Tessl Logo

manual-validation-harness

When validating a coralogix terraform-provider bug fix end-to-end against a real Coralogix env. Triggers on "test the fix locally", "reproduce the bug", "validate before merge".

78

1.98x
Quality

66%

Does it follow best practices?

Impact

99%

1.98x

Average score across 3 eval scenarios

SecuritybySnyk

Passed

No known issues

Optimize this skill with Tessl

npx tessl skill review --optimize ./.claude/skills/manual-validation-harness/SKILL.md
SKILL.md
Quality
Evals
Security

Manual end-to-end validation

Trigger: user wants to exercise a provider bug fix against a real env, post-build pre-merge.

Procedure:

  1. make install from the PR branch (builds + installs to ~/.terraform.d/plugins/locally/debug/coralogix/1.5/<arch>).
  2. Isolated workdir: mkdir /tmp/<ticket>-test && cd $_.
  3. Project-local terraformrc with dev_overrides for coralogix/coralogix pointing at the install dir above. export TF_CLI_CONFIG_FILE=$PWD/terraformrc.
  4. Multi-step scenarios: keep step files in ./steps/, cp steps/stepN.tf main.tf between runs. (Terraform reads every .tf in cwd → loose step files cause duplicate-resource errors.)
  5. Each step: terraform apply -auto-approve, verify in UI, then terraform plan → expect No changes. (idempotency catches perpetual-diff regressions).
  6. Cleanup: terraform destroy -auto-approve && rm -rf /tmp/<ticket>-test.

Gotchas:

  • 403 from API = key/env mismatch, not a provider bug.
  • "Provider development overrides are in effect" warning is expected.
  • Don't write .tf files via heredoc in chat — paste indentation breaks them. Use an editor or have Claude Write them.

Why: unit + acceptance tests catch regressions; manual validation gives the reviewer end-to-end confidence and exposes UI-layer behaviour the test suite can't observe.

Repository
coralogix/terraform-provider-coralogix
Last updated
Created

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.