CtrlK
BlogDocsLog inGet started
Tessl Logo

pantheon-ai/skill-quality-auditor

Audit and improve skill collections with an 8-dimension scoring framework, duplication detection, remediation planning, and CI quality gates; use when evaluating skill quality, generating remediation plans, validating report format, or enforcing repository-wide skill artifact conventions.

Does it follow best practices?

Evaluation93%

1.33x

Agent success when using this tile

Validation for skill structure

Overview
Skills
Evals
Files

reporting-analysis.mdreferences/

category:
reporting
priority:
MEDIUM
source:
skill audit analysis

Audit Report Analysis

Guide to interpreting skill quality audit reports and taking action based on findings.

Report Types

Quality Report (skill-audit-*.md)

Contains:

  • Overall skill collection statistics
  • Individual skill scores and grades
  • Grade distribution
  • Recommendations by priority

Duplication Report (duplication-report-*.md)

Contains:

  • Skill pairs with similarity percentages
  • High-priority consolidation candidates
  • Common content patterns
  • Remediation recommendations

Aggregation Plan (aggregation-plan-*.md)

Contains:

  • Source skills for consolidation
  • Proposed category structure
  • Estimated effort and ROI
  • Implementation steps

Analyzing Quality Reports

Grade Distribution

| Grade | Count | Percentage |
|-------|-------|------------|
| A     | 35    | 70%        |
| B     | 12    | 24%        |
| C     | 3     | 6%         |

Interpretation:

  • Target: 90%+ A-grade
  • Warning: Any C-grade skills
  • Critical: D or F grades

Score Analysis

| Skill | Score | Grade | Lines |
|-------|-------|-------|-------|
| bdd-testing | 98/120 | A | 64 |
| typescript-advanced | 85/120 | B | 450 |
| old-patterns | 72/120 | C | 800 |

Actions by Score:

ScorePriorityAction
108+MaintainKeep current, monitor
96-107LowMinor improvements optional
90-95MediumReview for improvements
78-89HighRequires attention
<78CriticalImmediate action needed

Common Score Patterns

High Lines, Low Score (450, 72):

  • Problem: Overly verbose, not focused
  • Solution: Progressive disclosure, extract to references

Low Lines, Low Score (80, 75):

  • Problem: Missing content, incomplete
  • Solution: Add missing dimensions (anti-patterns, examples)

High Lines, High Score (450, 105):

  • Problem: Good content but needs reorganization
  • Solution: Split into navigation hub + references

Analyzing Duplication Reports

Similarity Matrix

| Skill Pair | Similarity | Common Lines | Action |
|------------|------------|--------------|--------|
| bdd-gherkin ↔ cucumber | 42% | 287 | Aggregate |
| typescript-* family | 28% avg | 156 | Consider |

Action by Similarity:

SimilarityPriorityAction
>50%CriticalImmediate consolidation
35-50%HighPlan aggregation
20-35%MediumReview for opportunity
<20%LowMonitor

Family Analysis

## BDD Family (6 skills, 2,032 lines, 35% avg duplication)
- bdd-collaboration
- bdd-gherkin
- bdd-patterns
- bdd-principles
- bdd-scenarios
- cucumber-best-practices

Actions:

  • Calculate consolidation ROI
  • Plan aggregation implementation
  • Schedule remediation

Key Metrics to Track

Quality Metrics

MetricTargetCalculation
A-grade rate>90%A-count / total
Average score>100Total score / count
Max file size<500 lineswc -l SKILL.md
Description quality100%With keywords / total

Duplication Metrics

MetricTargetCalculation
Average duplication<5%Sum similarity / pairs
Critical pairs (>35%)0Count where >35%
Aggregation candidates<5Count where >20%

Maintenance Metrics

MetricTargetCalculation
Deprecated skillsMinimizeCount in .deprecated/
Orphan references0Broken @see links
Outdated reports<7 daysAge of latest audit

Action Planning

Immediate Actions (Critical Issues)

  1. Address C-grade skills

    • Read skill content
    • Identify missing dimensions
    • Add anti-patterns, examples
    • Re-evaluate
  2. Fix critical duplication

    • Consolidate >35% pairs
    • Follow aggregation implementation
    • Verify no breakage
  3. Fix broken references

    • Identify orphan @see links
    • Update or remove references
    • Verify skill still works

Short-term Actions (1-2 weeks)

  1. Improve B-grade skills

    • Add missing content
    • Improve descriptions
    • Add examples
  2. Plan aggregations

    • Identify families
    • Design category structure
    • Estimate effort
  3. Update documentation

    • Refresh AGENTS.md files
    • Update metrics
    • Document changes

Long-term Actions (Monthly)

  1. Review skill collection

    • Identify unused skills
    • Consider deprecation
    • Archive outdated content
  2. Update baselines

    • Re-run full audit
    • Compare with previous
    • Track trends
  3. Team review

    • Present findings
    • Get feedback
    • Prioritize improvements

Report Comparison

Tracking Changes Over Time

# Compare with previous audit
diff .context/analysis/skill-audit-2026-01-15.md \
     .context/analysis/skill-audit-2026-02-20.md

Look for:

  • Grade changes (improvements/regressions)
  • New skills added
  • Skills removed/deprecated
  • Score trends

Trend Indicators

TrendMeaning
A-grade increasingQuality improving
Average score risingSkills getting better
Duplication decreasingConsolidation working
File sizes shrinkingProgressive disclosure applied

Reporting to Stakeholders

Executive Summary

# Skill Quality Summary

## Overall Health: Good (87/100)

- **Skills**: 50 active, 10 deprecated
- **Quality**: 72% A-grade (target: 90%)
- **Duplication**: 12% (target: <5%)
- **Trend**: Improving (+5% from last quarter)

## Key Actions
1. Consolidate BDD family (6 → 1 skill)
2. Improve 8 B-grade skills to A
3. Reduce duplication from 12% to <5%

Technical Details

Provide full reports for technical teams with:

  • Complete score breakdowns
  • All dimension scores
  • Specific improvement recommendations
  • Code examples for fixes

See Also

  • reporting-dashboards.md - Visualization
  • advanced-trends-analysis.md - Historical tracking
  • scripts-audit-workflow.md - Running audits

Install with Tessl CLI

npx tessl i pantheon-ai/skill-quality-auditor

SKILL.md

tile.json