CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

mcp-oauth-triage

Use when an MCP connector that used to authenticate now fails (tool calls 401 after a good token exchange, breaks after refresh, returns another account's data), before changing code. Classifies the fault with oauth_triage.py.

Quality

82%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

81%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

An exceptionally actionable diagnostic skill: every primary command was verified to run and produce exactly the documented output, and the workflow's validation gates, feedback loops, and checklist are exemplary. The weaknesses are peripheral — a philosophy-heavy Architecture section, duplicated class descriptions between the body and the reference file, and a maintenance-harness path that is broken as shipped.

Suggestions

Ship `expected_outputs/verdicts.json` (or have run_fixtures.py generate it) so the documented `run_fixtures.py --verbose # expect 6/6` maintenance command actually executes — as bundled, it errors out.

Deduplicate the Class A–E descriptions: keep one-line discriminators plus fixes in SKILL.md and move the full mechanism prose to references/failure-classes.md, trimming the body by ~40 lines.

Compress the Architecture section to a few lines (or fold into the reference file) — it explains the tool's internal design rather than instructing the agent using it.

DimensionReasoningScore

Conciseness

The body is dense and high-signal — tables, exact commands, and per-class fixes with almost no re-explanation of OAuth basics Claude already knows. Minor over-explanation remains: the ~25-line Architecture section ("Detectors are independent", "Evidence is text") describes internal design philosophy rather than instructing the agent, and the Overview carries some narrative rhetoric ("A finding you cannot restate in words is not evidence") that could be trimmed. That matches anchor 4 ('efficient; minor instances of over-explanation') rather than 5.

4 / 5

Actionability

Usage commands are copy-paste ready and verified executable (running the classifier on a shipped fixture reproduces the documented output verbatim), with an exit-code table, real transcripts, and a per-class confirmation-before-fix protocol. However, the documented maintenance command `python3 scripts/run_fixtures.py --verbose` fails as shipped because the referenced `expected_outputs/verdicts.json` is missing from the bundle — a concrete gap in an otherwise fully executable skill, matching anchor 4 ('mostly executable; minor gaps').

4 / 5

Workflow Clarity

The four-step workflow (collect artifacts → run classifier → match class to fix → verify round trip) is explicitly sequenced with validation at every stage: confirmation before each fix because fixes are destructive to each other, exit-code semantics with "Exit 0 is not a health check" directing the reader back to Step 1 on a live symptom, "Two of three is not a fix", and a dedicated Verification Checklist. This is a textbook match for anchor 5 (explicit validation, feedback loops, checklist).

5 / 5

Progressive Disclosure

Bundle structure is real and well-organized: SKILL.md points to two one-level-deep reference files (both exist and deliver the promised decision table and verification recipes), plus scripts/ and assets/fixtures/ with 9 fixture files. Minor gaps keep it below 5: the five class descriptions are duplicated between SKILL.md and references/failure-classes.md rather than cleanly split, and the body cites `expected_outputs/verdicts.json`, which does not exist in the bundle — a broken navigation path.

4 / 5

Total

17

/

20

Passed

Description

73%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A high-quality description with an excellent explicit 'when' clause carrying concrete, natural trigger phrases and a well-isolated niche. The 'what' is clear but under-sells the skill's actual scope (fault classification plus class-specific fixes and verification), which caps specificity and completeness.

Suggestions

Expand the 'what' clause to name one or two more concrete capabilities, e.g. "Classifies the fault into five classes with oauth_triage.py and maps each class to its fix" — this would lift both specificity and completeness.

Add common user synonyms such as "unauthorized errors" or "auth keeps breaking" to the trigger list to broaden natural-term coverage.

State the regression framing exclusion ("not for first-time setup that never worked") in the description itself, not only in the body, to further reduce conflict with MCP setup skills.

DimensionReasoningScore

Specificity

The only stated action is "Classifies the fault with oauth_triage.py" — a single concrete action naming the domain (MCP connector OAuth regression) and the tool. The parenthetical enumerates concrete failure symptoms ("tool calls 401 after a good token exchange, breaks after refresh, returns another account's data"), but those are triggers, not capabilities, so coverage of what the skill does is not comprehensive. This matches anchor 3 ("names domain and 1-2 concrete actions") rather than 4 ("lists several specific actions"), and exceeds 2 because the action is specific and tool-named rather than generic.

3 / 5

Completeness

Both halves are explicitly present: "Use when an MCP connector that used to authenticate now fails (...)" is an exemplary, highly specific 'when' with concrete trigger phrases, and "Classifies the fault with oauth_triage.py" is a clear 'what'. It falls short of anchor 5 because the 'what' is a single terse clause — the skill's class-to-fix mapping and verification loop are unmentioned — leaving the stated capability thinner than the 'when'.

4 / 5

Trigger Term Quality

Strong natural phrases a user debugging this would actually say: "tool calls 401", "breaks after refresh", "returns another account's data", "used to authenticate now fails", "MCP connector". Missing common synonyms such as "unauthorized", "auth loop", or "token expired", which keeps it below the comprehensive anchor 5 but comfortably above anchor 3's 'some relevant keywords'.

4 / 5

Distinctiveness Conflict Risk

"An MCP connector that used to authenticate now fails" carves out a clear niche with distinct triggers (401 after good exchange, post-refresh failure, wrong-account data) that no generic auth or MCP-setup skill would claim. Conflict risk is minimal; first-time-setup problems, the nearest neighboring domain, are explicitly excluded in the body.

5 / 5

Total

16

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

alirezarezvani/claude-skills

Path

engineering/skills/mcp-oauth-triage/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.