manual-validation-harness
When validating a coralogix terraform-provider bug fix end-to-end against a real Coralogix env. Triggers on "test the fix locally", "reproduce the bug", "validate before merge".
78
66%
Does it follow best practices?
Impact
99%
1.98xAverage score across 3 eval scenarios
Passed
No known issues
Optimize this skill with Tessl
npx tessl skill review --optimize ./.claude/skills/manual-validation-harness/SKILL.mdEvaluation results
100%
54%Automate Provider Validation Workflow
Validation workflow automation script
make install step
0%
100%
Isolated /tmp workdir
0%
100%
terraformrc file creation
70%
100%
dev_overrides provider path
100%
100%
TF_CLI_CONFIG_FILE export
80%
100%
steps/ subdirectory
100%
100%
cp steps pattern
0%
100%
terraform apply -auto-approve
100%
100%
Idempotency plan check
0%
100%
Cleanup destroy + rm
35%
100%
97%
52%Set Up a Terraform Provider Validation Workspace
Multi-step test directory structure
steps/ subdirectory used
0%
100%
No loose .tf in root
100%
100%
main.tf as active file
0%
70%
terraformrc with dev_overrides
66%
100%
Provider source in dev_overrides
100%
100%
TF_CLI_CONFIG_FILE documented
100%
100%
cp steps pattern documented
0%
100%
Idempotency check documented
0%
100%
100%
41%Debug a Broken Provider Validation Script
Diagnose validation setup issues
Absolute TF_CLI_CONFIG_FILE path
100%
100%
dev_overrides warning classified correctly
100%
100%
403 classified as env/key issue
100%
100%
Loose .tf files moved to steps/
0%
100%
cp steps/stepN.tf main.tf pattern added
80%
100%
Idempotency check added
0%
100%
Cleanup includes rm -rf
50%
100%
- Repository
- coralogix/terraform-provider-coralogix
- Commit
f197371
- Evaluated
- Agent
- Claude Code
- Model
- Claude Sonnet 4.6
Table of Contents
Is this your skill?
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.