manual-validation-harness
When validating a coralogix terraform-provider bug fix end-to-end against a real Coralogix env. Triggers on "test the fix locally", "reproduce the bug", "validate before merge".
78
66%
Does it follow best practices?
Impact
99%
1.98xAverage score across 3 eval scenarios
Passed
No known issues
Optimize this skill with Tessl
npx tessl skill review --optimize ./.claude/skills/manual-validation-harness/SKILL.mdQuality
Discovery
47%Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.
The description clearly identifies a narrow niche (Coralogix terraform-provider bug fix validation) and provides explicit trigger phrases, which helps with distinctiveness and trigger matching. However, it fails to describe what concrete actions the skill performs, leaving the 'what' dimension very weak. The description reads more like a trigger condition than a capability summary.
Suggestions
Add specific concrete actions the skill performs, e.g., 'Configures local terraform provider overrides, runs acceptance tests, applies terraform plans against a real Coralogix environment, and verifies resource state to validate bug fixes.'
Expand trigger terms to include common variations like 'terraform acceptance test', 'provider dev override', 'TF provider testing', 'run tests locally'.
| Dimension | Reasoning | Score |
|---|---|---|
Specificity | The description mentions 'validating a coralogix terraform-provider bug fix end-to-end' but does not list any concrete actions (e.g., run terraform plan, apply config, check resource state). It describes a general scenario rather than specific capabilities. | 1 / 3 |
Completeness | The 'when' is addressed with explicit trigger phrases, but the 'what' is very weak — it only says 'validating a bug fix end-to-end' without explaining what concrete steps or actions the skill performs. The 'what' is essentially missing substantive detail. | 2 / 3 |
Trigger Term Quality | Includes some natural trigger phrases like 'test the fix locally', 'reproduce the bug', 'validate before merge', and domain terms like 'coralogix', 'terraform-provider'. However, it misses common variations like 'terraform apply', 'acceptance test', 'provider testing', or 'TF provider'. | 2 / 3 |
Distinctiveness Conflict Risk | The description is highly specific to a niche domain — Coralogix terraform-provider bug fix validation. It is very unlikely to conflict with other skills due to the narrow scope and specific product/tool references. | 3 / 3 |
Total | 8 / 12 Passed |
Implementation
85%Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
This is a well-crafted, concise skill that clearly sequences a manual validation workflow with appropriate checkpoints. Its main weakness is the lack of a concrete terraformrc example block, which is a non-obvious configuration that Claude would benefit from seeing verbatim. The gotchas section adds genuine value by preempting common misdiagnoses.
Suggestions
Add a concrete, copy-paste-ready example of the terraformrc file with dev_overrides block, since the exact syntax is easy to get wrong.
Include a minimal example .tf step file (e.g., steps/step1.tf) to make the multi-step scenario pattern fully executable.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | Every sentence earns its place. No unnecessary explanations of what Terraform is or how providers work. The 'Why' section at the end is brief and justified. The gotchas are terse and high-value. | 3 / 3 |
Actionability | The procedure gives concrete commands (make install, terraform apply, etc.) and specific paths, but lacks executable examples for the terraformrc content and the step file structure. The dev_overrides block is described but not shown, which is a key detail Claude would need to produce correctly. | 2 / 3 |
Workflow Clarity | Clear 6-step sequence with explicit validation checkpoints (terraform plan expecting 'No changes' for idempotency, verify in UI). Includes cleanup step and explains why multi-step scenarios need the steps/ directory pattern to avoid duplicate-resource errors. The feedback loop is implicit but clear: if plan doesn't show 'No changes', there's a regression. | 3 / 3 |
Progressive Disclosure | For a simple, single-purpose skill under 50 lines with no need for external references, the content is well-organized into clear sections (Trigger, Procedure, Gotchas, Why). No bundle files are needed for this scope. | 3 / 3 |
Total | 11 / 12 Passed |
Validation
100%Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.
Validation — 11 / 11 Passed
Validation for skill structure
No warnings or errors.
- Repository
- coralogix/terraform-provider-coralogix
- Commit
f197371
- Reviewed
Table of Contents
Is this your skill?
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.