CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/openapi-contract-diff

Diffs two OpenAPI versions for breaking changes (removed endpoints, deleted schemas, narrowed enums, response shape changes) using `oasdiff breaking`, severity-classifies the findings (ERR / WARN / INFO), and gates CI with `--fail-on`. Use when reviewing an OpenAPI spec change in a PR or on a release tag.

72

Quality

90%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

82%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-organized, highly actionable skill body: install, run, read, and gate instructions are all copy-paste executable, and the CI integration section is complete enough to adopt directly. The main improvement levers are trimming the repeated citation asides and pact-comparison prose, and moving the long CI workflow example into a reference file.

DimensionReasoningScore

Conciseness

The body is mostly lean: severity tiers as a compact table, executable command blocks with one-line commentary, and no explanations of what OpenAPI is or how diffing works. A few spots could be trimmed — the repeated '(Per [oasdiff-readme][readme])' citation lines and the pact-comparison prose in 'Overview'/'Authoring' partially restate each other. This lands on anchor 4 ('efficient; minor instances of over-explanation that could be trimmed') rather than 5, and is clearly above anchor 3 because nothing teaches Claude concepts it already knows.

4 / 5

Actionability

Every section gives copy-paste-ready commands: `go install`/`docker pull` install, four `--format` invocations, a Docker run variant, `--fail-on` gate examples, a `jq` triage pipeline, and a complete GitHub Actions workflow with merge-base spec extraction and artifact upload. Example report output (text and JSON) is included so Claude knows what to expect. This matches anchor 5 ('fully executable; copy-paste ready; specific examples cover the common cases'), not 4, since there are no significant gaps.

5 / 5

Workflow Clarity

The sequence Install → Running (local/remote/Docker) → Reading the report → CI integration is coherent and each step builds on the prior. The exit code is explicitly established as the gate signal ('The exit code is the gate signal') and the CI workflow includes a real checkpoint ('if: always()' artifact upload plus `--fail-on ERR`). This is a read-only review tool, so the destructive/batch validation cap does not apply. It sits at anchor 4 rather than 5 because there is no explicit error-recovery loop (e.g., what to do when the gate fails besides the one `x-extensible-enum` mitigation) and no pre-flight verification step; it is above anchor 3 because the checkpoints that do exist are explicit, not implicit.

4 / 5

Progressive Disclosure

No bundle files exist (references/, scripts/, assets/ are absent), so all content is inline in SKILL.md, organized under clear headers (Overview, When to use, Install, Running, What counts as breaking, Reading the report, CI integration, References). External references are clearly signaled and one level deep via a link table. Anchor 4 fits: good structure, content mostly appropriately placed, with a minor gap — the ~35-line inline GitHub Actions workflow and the full JSON report example are borderline candidates for a references file that would keep SKILL.md as a pure overview. Not 5 because the body leans toward monolithic; not 3 because navigation is easy and nothing is buried.

4 / 5

Total

17

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it names the tool, enumerates concrete breaking-change categories and severity tiers, states the CI-gating capability, and closes with an explicit 'Use when...' trigger clause. The only weakness is trigger-term coverage missing common synonyms such as 'Swagger' and spec file extensions like 'openapi.yaml'.

DimensionReasoningScore

Specificity

The description lists multiple concrete, specific actions with concrete artifacts: 'Diffs two OpenAPI versions for breaking changes (removed endpoints, deleted schemas, narrowed enums, response shape changes) using `oasdiff breaking`', 'severity-classifies the findings (ERR / WARN / INFO)', and 'gates CI with `--fail-on`'. Coverage of the skill's capability surface is comprehensive, matching the anchor for listing multiple specific concrete actions. Not 4 because there are no meaningful gaps in the action list; not below because nothing is generic or abstract.

5 / 5

Completeness

Both questions are explicitly answered: the 'what' is concrete ('Diffs two OpenAPI versions... severity-classifies the findings (ERR / WARN / INFO), and gates CI with `--fail-on`') and the 'when' is an explicit trigger clause ('Use when reviewing an OpenAPI spec change in a PR or on a release tag'). This matches anchor 5 exactly; it is above anchor 4 because the when-clause is explicit and specific rather than merely present.

5 / 5

Trigger Term Quality

Good natural-keyword coverage: users would plausibly say 'OpenAPI', 'breaking changes', 'spec change', 'PR', and 'release tag', all of which appear verbatim ('Use when reviewing an OpenAPI spec change in a PR or on a release tag'). A few natural terms are missing — most notably the common synonym 'Swagger', file extensions like 'openapi.yaml', and phrasings like 'API diff'. This matches anchor 4 ('good keyword coverage; a few natural terms missing') rather than 5's requirement of comprehensive synonyms and extensions, and is well above anchor 3's 'missing common variations'.

4 / 5

Distinctiveness Conflict Risk

The description carves a clear niche — OpenAPI spec diffing with a named tool (`oasdiff breaking`), severity tiers (ERR / WARN / INFO), and a CI gate flag — that no generic description would collide with. It even implicitly disambiguates from consumer-driven contract testing by framing the trigger around spec review in PRs. Minimal conflict risk; matches anchor 5 rather than 4's 'minor overlap risk with closely related skills'.

5 / 5

Total

19

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

Table of Contents