pantheon-ai/skill-quality-auditor
Audit and improve skill collections with a 9-dimension scoring framework (Knowledge Delta, Mindset, Anti-Patterns, Specification Compliance, Progressive Disclosure, Freedom Calibration, Pattern Recognition, Practical Usability, Eval Validation), duplication detection, remediation planning, baseline comparison, and CI quality gates; use when evaluating skill quality, generating remediation plans, detecting duplicates, validating artifact conventions, or enforcing publication thresholds.
93
89%
Does it follow best practices?
Impact
99%
1.26xAverage score across 5 eval scenarios
Passed
No known issues
task.mdevals/scenario-5/
Task: Skills Portfolio Duplication Analysis
A skills collection contains the following 8 skills in skills/:
ci-cd/github-actions-generatorci-cd/github-actions-validatorci-cd/gitlab-ci-generatorci-cd/gitlab-ci-validatorinfrastructure/terraform-generatorinfrastructure/terraform-validatorinfrastructure/terragrunt-generatorinfrastructure/terragrunt-validator
The team suspects that the generator/validator pairs within each tool have high content overlap, and that terraform and terragrunt skills may also overlap significantly.
Analyse the collection for duplication and produce a consolidation recommendation report.
Output Specification
Produce:
- similarity-analysis.md — pairwise similarity percentages for all relevant pairs
- consolidation-recommendations.md — which skills to consolidate and how (Navigation Hub pattern)
- duplication-report.json — structured data: pairs, similarity %, action (monitor/plan/immediate)
Context
Similarity thresholds:
- <20%: monitor only
- 20–35%: plan aggregation
-
35%: immediate action required
assets
evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
references
scripts