CtrlK
BlogDocsLog inGet started
Tessl Logo

pantheon-ai/skill-quality-auditor

Audit and improve skill collections with an 8-dimension scoring framework, duplication detection, remediation planning, and CI quality gates; use when evaluating skill quality, generating remediation plans, validating report format, or enforcing repository-wide skill artifact conventions.

Does it follow best practices?

Evaluation93%

1.33x

Agent success when using this tile

Validation for skill structure

Overview
Skills
Evals
Files

remediation-planning.mdreferences/

Remediation Planning Guide

Guide for creating remediation plans from skill quality audit results.

When to Use This Guide

Use this guide when:

  • A skill has been audited and scored below target (typically <108/120, grade B or lower)
  • You need to create an actionable plan to improve skill quality
  • Converting audit findings into structured improvement tasks

Prerequisites

  1. Completed Audit: Run sh skills/skill-quality-auditor/scripts/evaluate.sh <skill-name> --json
  2. Audit Report: Location should be .context/audits/<skill-name>-audit-YYYY-MM-DD.md
  3. Template: Load skills/skill-quality-auditor/templates/remediation-plan-template.yaml

Standard Remediation Plan Format

All remediation plans follow this structure:

Executive Summary

FieldDescription
Current ScoreActual score from audit (e.g., 84/120, 70%)
Target ScoreDesired score (typically +15-20 points)
Current/Target GradeLetter grade (F/D/C/B/A)
PriorityCritical/High/Medium/Low
EffortS/M/L estimate

Focus Areas: Top 2-3 dimensions requiring attention

Verdict: One-line summary: "Priority improvements required" vs "Targeted improvements recommended"

Critical Issues Table

Identify the top 3-5 issues from the audit:

IssueDimensionSeverityImpact
DescriptionD# (score/max)Critical/High/Medium/LowWhy this matters

Prioritize by:

  1. Critical/High severity
  2. Dimensions with lowest scores
  3. Issues blocking other improvements

Detailed Remediation Steps

For each priority dimension:

### Phase N: [Dimension Name] - Priority: [Priority]

**Target**: Increase from [current]/[max] to [target]/[max] (+[delta] points)

#### Step N.1: [Specific Action]

**[File to modify]**: `path/to/file.md`

[Code example if applicable]

Code Block Escaping

When documenting code examples that contain markdown code fences, use 4 backticks:

**BAD**:
```markdown
Some markdown content here
```

**GOOD**:
```markdown
Good content here
```

Success Criteria

Measurable targets for verification:

CriterionMeasurement
Dimension Score>= target/max
Overall Score>= target-score (target-grade)
References Created>= N files

Effort Estimate

PhaseEffortTime
Phase 1S/M/LHH hours
Phase 2S/M/LHH hours
TotalM/LX hours

Additional Sections

  • Dependencies: Any prerequisites or blockers
  • Rollback Plan: Git command to revert changes
  • Notes: Honest assessment of plan quality (rating out of 10)

Honest Rating Guidelines

Rate plans based on:

RatingCriteria
9-10/10Template gold standard, comprehensive
8/10Follows Format B well, detailed code examples
7/10Good structure but gaps in specifics
6/10Major restructuring needed
<5/10Fundamental issues with approach

Common Remediation Patterns

Pattern 1: Anti-Pattern Addition

For D3 (Anti-Pattern Quality) improvements:

## Anti-Patterns

### NEVER: [Action]

**WHY**: [Reason]

**BAD**:
```[language]
[bad code]
```

**GOOD**:
```[language]
[good code]
```

Pattern 2: Reference Creation

For D5 (Progressive Disclosure) improvements:

  1. Create skills/<skill>/references/ directory
  2. Extract detailed content from SKILL.md
  3. Add navigation table to SKILL.md:
## Quick Reference

| Topic | Reference |
|-------|-----------|
| Topic Name | [references/topic.md](references/topic.md) |

Pattern 3: Trigger Expansion

For D7 (Pattern Recognition) improvements:

Update frontmatter description:

---
name: skill-name
description: |
  [Core description]. Use when: [trigger phrases].
  
  Keywords: [comma-separated keywords]
---

Verification

After implementing remediation:

sh skills/skill-quality-auditor/scripts/evaluate.sh <skill-name> --json
bunx markdownlint-cli2 "skills/<skill-name>/**/*.md"

Compare new score against target in plan.

Related References

  • framework-skill-judge-dimensions.md - Understanding the 8 dimensions
  • framework-scoring-rubric.md - How scores are calculated
  • aggregation-pattern.md - For larger consolidation efforts

Install with Tessl CLI

npx tessl i pantheon-ai/skill-quality-auditor

SKILL.md

tile.json