CtrlK
BlogDocsLog inGet started
Tessl Logo

pantheon-ai/skill-quality-auditor

Audit and improve skill collections with an 8-dimension scoring framework, duplication detection, remediation planning, and CI quality gates; use when evaluating skill quality, generating remediation plans, validating report format, or enforcing repository-wide skill artifact conventions.

Does it follow best practices?

Evaluation93%

1.33x

Agent success when using this tile

Validation for skill structure

Overview
Skills
Evals
Files

aggregation-pattern.mdreferences/

Skill Aggregation Pattern: Progressive Disclosure for Large Skill Collections

Pattern Name: Navigation Hub + References
Origin: supabase-postgres-best-practices (108/120 - A grade)
Use Case: Consolidating related skills to reduce redundancy and improve usability
Impact: 96%+ reduction in main file size, <5% duplication, A+ average grades


Problem Statement

Large skill collections face several challenges:

  1. Duplication - Similar content repeated across multiple skills (35%+ redundancy)
  2. Discoverability - Users unsure which skill to load for their task
  3. Context Bloat - Loading 600+ line skills when only 100 lines needed
  4. Maintenance Burden - Updating concepts in multiple places
  5. Quality Issues - Skills with 40% tutorial content score C grade

Example:

  • 6 BDD skills (1,839 lines total) with 35% duplication
  • Users confused: "Should I use bdd-gherkin or cucumber-best-practices?"
  • Updates required in 3 files for one concept change

Solution: Aggregation Pattern

Architecture

skill-name/
├── SKILL.md              # 60-90 line navigation hub
├── AGENTS.md             # Complete reference guide
└── references/           # Detailed content (100-500 lines each)
    ├── category1-topic1.md
    ├── category1-topic2.md
    ├── category2-topic1.md
    └── ...

Three Components

1. SKILL.md (Navigation Hub)

  • 60-90 lines maximum
  • Overview (2-3 sentences)
  • "When to Apply" section
  • Priority-based category table (CRITICAL/HIGH/MEDIUM/LOW)
  • Progressive disclosure instructions
  • External references

2. AGENTS.md (Reference Guide)

  • Skill structure overview
  • Usage workflow
  • Complete category table with descriptions
  • Full listing of all reference files by category
  • File count summary
  • Navigation best practices

3. references/ (Expert Content)

  • One file per focused topic
  • Organized by category prefix (e.g., principles-, patterns-)
  • 100-500 lines per file (only when justified)
  • Frontmatter: category, priority, source
  • Expert knowledge only, no tutorials
  • Best practices + anti-patterns
  • Code examples
  • Related references (@see links)

Implementation Steps

1. Identify Consolidation Candidates

Look for:

  • Families of related skills (e.g., BDD, TypeScript, Bun)
  • High duplication (>20% shared content)
  • User confusion ("Which skill should I use?")
  • Low grades (multiple C-grade skills in same domain)
  • Oversized skills (>500 lines)

Example:

BDD family: 6 skills, 1,839 lines, 35% duplication → Consolidate
Random unrelated skills → Keep separate

2. Design Category Structure

Organize by:

  1. Priority - CRITICAL (foundations) → LOW (advanced/optional)
  2. Workflow - Natural progression through content
  3. Topic - Logical grouping of related concepts

Example - BDD Testing:

CRITICAL: principles-    (philosophy, Three Amigos)
HIGH:     gherkin-       (syntax, step definitions)
HIGH:     patterns-      (Given-When-Then, scenarios)
MEDIUM:   collaboration- (Example Mapping, workshops)
MEDIUM:   practices-     (best practices, anti-patterns)

3. Create Navigation Hub (SKILL.md)

Template:

---
name: skill-name
description: [Comprehensive with WHAT, WHEN, KEYWORDS]
consolidates: [list of original skills]
original_lines: XXXX
hub_lines: XX
reduction: XX%
---

# Skill Title

[2-3 sentence overview]

## When to Apply
- [Specific trigger scenarios]
- [Use case examples]

## Categories by Priority

| Priority | Category | Impact | Files |
|----------|----------|--------|-------|
| CRITICAL | [category] | [why critical] | X |
| HIGH     | [category] | [why high] | X |
| MEDIUM   | [category] | [why medium] | X |

## How to Use This Skill

1. Read this navigation hub first (60 lines)
2. Check AGENTS.md for complete file listing
3. Load specific reference files as needed:
   - For [use case] → `references/category-file.md`
   - For [use case] → `references/category-file.md`

## References
- [External links]

4. Create Reference Guide (AGENTS.md)

Template:

# [Skill Name] - Reference Guide

## Overview

This skill consolidates X original skills (X,XXX lines) into:
- Navigation hub: XX lines
- Reference files: XX files across X categories

## Usage Workflow

1. Load SKILL.md for overview
2. Identify your use case
3. Load relevant reference files by category
4. Work through categories by priority

## Category Guide

### CRITICAL Priority
- **[category]-** - [Description]
  - [file1.md] - [Purpose]
  - [file2.md] - [Purpose]

### HIGH Priority
[repeat structure]

## All Reference Files

[Complete alphabetical listing]

## File Count
Total: XX references across X categories

5. Extract Content to References

For each reference file:

  1. Read source skill and identify focused topic

  2. Extract expert knowledge (remove tutorials)

  3. Create reference file with frontmatter:

    ---
    category: [category-name]
    priority: CRITICAL|HIGH|MEDIUM|LOW
    source: [original-skill-name]
    ---
    
    # [Topic Title]
    
    [2-3 sentence overview]
    
    ## [Section]
    [Content with code examples]
    
    ## Best Practices
    [Actionable guidelines]
    
    ## Common Pitfalls
    [Anti-patterns with WHY]
    
    ## Related
    - @see references/[related-file].md
  4. Follow size targets:

    • Simple concepts: 100-200 lines
    • Complex patterns: 300-500 lines
    • Only exceed 500 lines when absolutely justified

6. Move Originals to .deprecated/

# Create .deprecated directory
mkdir -p .agents/skills/.deprecated

# Move consolidated skills
mv .agents/skills/original-skill .agents/skills/.deprecated/

# Create README explaining consolidation
cat > .agents/skills/.deprecated/README.md <<EOF
# Deprecated Skills

These skills consolidated into: [aggregation-skill-name]

See .agents/skills/[aggregation-skill-name]/ for new structure.
EOF

Quality Standards

Navigation Hub (SKILL.md)

Must Have:

  • 60-90 lines (exceptions rare)
  • Clear description with keywords
  • Priority-based categories
  • Progressive disclosure instructions
  • "When to Apply" section

Must Not Have:

  • Tutorial content
  • Code examples (move to references/)
  • Detailed explanations (move to references/)
  • Duplication from references

Reference Files

Must Have:

  • Proper frontmatter
  • Focused on ONE topic
  • Expert knowledge only
  • Code examples
  • Best practices + anti-patterns
  • Related references (@see)

Must Not Have:

  • Multiple unrelated topics
  • Tutorial content for basics
  • Duplication across files
  • Content AI assistants already know

AGENTS.md

Must Have:

  • Complete file listing
  • Category descriptions
  • Usage workflow
  • File count summary

Must Not Have:

  • Actual content (link to references/)
  • Tutorial explanations

Success Metrics

Before Aggregation

  • Multiple related skills (3-6)
  • 1,000-3,000+ total lines
  • 20-35% duplication
  • User confusion about which skill to use
  • C/B grade average (75-85/120)

After Aggregation

  • Single navigation hub
  • 60-90 line SKILL.md
  • <5% duplication
  • Clear category-based navigation
  • A/A+ grade (90-98/120)

Specific Improvements

MetricBeforeAfterImprovement
Main file size500-700 lines60-90 lines85-95%
Duplication20-35%<5%15-30pp
Skills count3-6166-83%
Average gradeC/BA/A+5-15 points
User confusionHighLowQualitative

Real-World Results

Case Study: BDD Testing Aggregation

Before:

  • 6 separate skills: bdd-collaboration, bdd-gherkin, bdd-patterns, bdd-principles, bdd-scenarios, cucumber-best-practices
  • 1,839 total lines
  • 35% duplication (Gherkin syntax in 3 skills)
  • Users load wrong skill 40% of time
  • Average grade: 90/120 (B)

After:

  • 1 aggregation: bdd-testing
  • 64-line navigation hub
  • 42 reference files planned (7 created)
  • <5% duplication
  • Clear category-based navigation
  • Grade: 98/120 (A+)

Results:

  • 96.5% reduction in main file size (1,839 → 64)
  • 86% reduction in duplication (35% → <5%)
  • 8-point grade improvement (90 → 98)
  • 100% user navigation clarity

Common Pitfalls

1. Over-Consolidation

Problem: Consolidating unrelated skills because they share technology
Example: Consolidating all "Python" skills regardless of domain
Solution: Only consolidate skills users would naturally use together

2. Under-Consolidation

Problem: Creating too many small aggregations
Example: Separate aggregations for "BDD Principles" and "BDD Patterns"
Solution: Consolidate related concepts into single aggregation with categories

3. Hub Too Large

Problem: Navigation hub exceeds 100 lines
Example: Including code examples, detailed explanations in SKILL.md
Solution: Move ALL detailed content to references/

4. Missing Priorities

Problem: All categories marked CRITICAL
Example: No clear order for learning progression
Solution: Use CRITICAL (foundations), HIGH (core), MEDIUM (specialized), LOW (advanced)

5. Poor Category Design

Problem: Categories not aligned with user workflow
Example: Alphabetical instead of logical progression
Solution: Design categories by priority and natural learning/usage flow

6. Reference Files Too Large

Problem: 800+ line reference files defeating progressive disclosure
Example: Single "typescript-everything.md" file
Solution: Split into focused 100-500 line files per topic

7. Incomplete AGENTS.md

Problem: Missing file listings, unclear navigation
Example: "See references/ for more" without specific guidance
Solution: List every reference file with description


When NOT to Aggregate

Don't aggregate if:

  1. Skills are unrelated - Technology overlap ≠ domain overlap
  2. No duplication - Each skill has unique content
  3. Different audiences - Beginner vs. expert skills
  4. Already optimal - Skill is 100-200 lines, A grade, clear focus
  5. Single skill in domain - No family of related skills

Example:

  • ✅ Aggregate: 6 BDD skills (same domain, same audience, high duplication)
  • ❌ Don't aggregate: proof-of-work + code-reviewer (different purposes despite both about code quality)

Maintenance

Adding New Content

  1. Create new reference file in appropriate category
  2. Update AGENTS.md with file listing
  3. Update SKILL.md category table if new category
  4. Maintain priority-based organization

Updating Content

  1. Update specific reference file (single source of truth)
  2. Update AGENTS.md if file renamed/moved
  3. Never update navigation hub with detailed content

Deprecating Content

  1. Move reference file to references/.archived/
  2. Remove from AGENTS.md listing
  3. Update related references (@see links)

Quarterly Audits

  1. Check for new skills that should be consolidated
  2. Verify no duplication creeping in
  3. Review priority assignments
  4. Check if any reference files should be split/merged
  5. Re-run skill-judge evaluation

Tools and Automation

Manual Process (Current)

  1. Identify consolidation candidates
  2. Read all source skills
  3. Design category structure
  4. Extract content manually
  5. Create reference files
  6. Write SKILL.md and AGENTS.md

Time: 4-6 hours per aggregation

Semi-Automated Process (Recommended)

  1. Run duplication detector script
  2. Generate consolidation candidates
  3. Review and approve category structure
  4. Auto-extract content sections
  5. Human review and refine
  6. Auto-generate AGENTS.md

Time: 2-3 hours per aggregation

Script Opportunities

# Duplication detector
./scripts/detect-skill-duplication.sh

# Category suggester
./scripts/suggest-categories.sh skill1 skill2 skill3

# Content extractor
./scripts/extract-to-references.sh source-skill target-aggregation

# AGENTS.md generator
./scripts/generate-agents-md.sh aggregation-dir

References

Pattern Origin

  • supabase-postgres-best-practices (108/120 - A grade)
  • 65-line hub managing 31 reference files
  • Perfect progressive disclosure implementation

Related Patterns

  • Progressive Disclosure (UX design)
  • Information Architecture (IA)
  • Hub-and-Spoke (knowledge management)
  • Single Source of Truth (SSOT)

External Resources

  • Progressive Disclosure - Nielsen Norman Group
  • Information Architecture Basics - Usability.gov

Conclusion

The Navigation Hub + References pattern successfully:

  • ✅ Reduces main file size by 96%+
  • ✅ Eliminates 80-90% of duplication
  • ✅ Improves grades from B to A+
  • ✅ Provides clear navigation
  • ✅ Enables progressive disclosure
  • ✅ Maintains single source of truth
  • ✅ Scales to large skill collections

Use this pattern when consolidating 3+ related skills with >20% duplication.

Key Principle: Navigation hub answers "what and where," references provide "how and why."

Install with Tessl CLI

npx tessl i pantheon-ai/skill-quality-auditor@0.1.4

SKILL.md

tile.json