CtrlK
BlogDocsLog inGet started
Tessl Logo

pantheon-ai/skill-quality-auditor

Audit and improve skill collections with a 9-dimension scoring framework (Knowledge Delta, Mindset, Anti-Patterns, Specification Compliance, Progressive Disclosure, Freedom Calibration, Pattern Recognition, Practical Usability, Eval Validation), duplication detection, remediation planning, baseline comparison, and CI quality gates; use when evaluating skill quality, generating remediation plans, detecting duplicates, validating artifact conventions, or enforcing publication thresholds.

93

1.26x
Quality

89%

Does it follow best practices?

Impact

99%

1.26x

Average score across 5 eval scenarios

SecuritybySnyk

Passed

No known issues

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-1/

{
  "context": "Tests whether the agent applies the 9-dimension framework correctly to a skill heavy with redundant content (SQL basics, installation steps, generic best-practices) and produces a scored report with actionable remediation.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "9-dimension framework applied",
      "description": "Uses all 9 dimensions: Knowledge Delta, Mindset+Procedures, Anti-Pattern Quality, Specification Compliance, Progressive Disclosure, Freedom Calibration, Pattern Recognition, Practical Usability, Eval Validation",
      "max_score": 12
    },
    {
      "name": "Redundant content identified",
      "description": "Flags SQL basics, installation instructions, and generic best-practices as redundant content not worth expert attention",
      "max_score": 15
    },
    {
      "name": "Knowledge Delta scored low",
      "description": "Assigns D1 a low score (<=10/20) reflecting the high redundancy ratio of the input skill",
      "max_score": 12
    },
    {
      "name": "Numerical scores per dimension",
      "description": "Provides specific numerical scores for each dimension with a brief justification",
      "max_score": 15
    },
    {
      "name": "A-grade threshold referenced",
      "description": "States the A-grade target as >=126/140 or equivalent percentage",
      "max_score": 10
    },
    {
      "name": "Actionable remediation steps",
      "description": "Remediation plan contains specific file-level changes (what to add/remove/rewrite) with S/M/L effort sizing",
      "max_score": 20
    },
    {
      "name": "Specification compliance issues noted",
      "description": "Identifies the weak description field ('Help with SQL queries.') as a D4 compliance failure",
      "max_score": 8
    },
    {
      "name": "Progressive disclosure gap noted",
      "description": "Notes absence of references/ directory and/or content frontloading as a D5 weakness",
      "max_score": 8
    }
  ]
}

SKILL.md

tile.json