check-oas

Detect breaking changes in docs/mapi/openapi.yaml and check whether the committed spec is stale; run the OAS checks, interpret findings, and guide fixes.

0.97x

Quality

83%

Does it follow best practices?

Impact

82%

0.97x

1 of 3 eval scenarios. Add 2 more for a full score.

Securityby

Medium

Suggest reviewing before use

Quality

Content

90%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

An exemplary operational skill: concrete, executable commands with exit-code-driven decision points, error-recovery guidance via a fix table, and dense project-specific knowledge with zero filler. The only weaknesses are minor — no explicit re-run step in the fix loops, and all reference material inlined in a single file rather than progressively disclosed.

Dimension	Reasoning	Score
Conciseness	Lean and dense throughout: comparison tables, exit-code semantics, and copy-paste commands with no padding and no explanation of concepts Claude already knows. Project-specific details (sorted output in `GraviteeApiDefinition.afterScan()`, `info.version` patching, license-header reattachment) are exactly the things Claude could not know.	5 / 5
Actionability	Every workflow is fully executable: exact bash invocations for both checks (CI and local `--also-make` variants), oasdiff install for macOS/Linux pinned to `.oasdiff-version`, all four `--base` ref forms, `--use-head false`, direct oasdiff usage, and both regeneration paths. Copy-paste ready with the common cases covered.	5 / 5
Workflow Clarity	Clear sequences with real validation checkpoints: exit-code semantics, error-vs-warning interpretation of oasdiff output, and an error→fix table for recovery. Falls short of 5 only because the fix loops never explicitly say to re-run the check after applying a fix and committing (e.g. after `regen-oas.sh`, re-run the staleness check), leaving a minor checkpoint gap.	4 / 5
Progressive Disclosure	No bundle files exist (references/, scripts/, assets/ are absent), so all content lives in a well-sectioned SKILL.md — clear headings, a summary table up top, and one internal anchor link that resolves. Anchor 4 fits: structure is good and content placement is appropriate for a skill this size, but reference-style material (the breaking-change category table and oasdiff output interpretation) is inlined rather than split out, and the body leans on project files (`.circleci/scripts/*`, `scripts/regen-oas.sh`) rather than bundled ones.	4 / 5
	Total	18 / 20 Passed

Description

70%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific, action-oriented, and clearly distinct, but it answers only "what" — there is no explicit "when to use this skill" guidance, and it under-represents the regeneration workflow the body teaches. Adding a 'Use when...' clause would raise completeness to the level of the other dimensions.

Suggestions

Add an explicit trigger clause, e.g. "Use when modifying the management API REST layer, changing JAX-RS annotations, or when CI reports the spec is stale or has breaking changes."

Mention spec regeneration (`scripts/regen-oas.sh`) since committing a regenerated spec is the primary remediation path described in the body.

Add a natural synonym or two (e.g. "OpenAPI/Swagger spec", "API diff") to broaden trigger-term coverage.

Dimension	Reasoning	Score
Specificity	Lists several concrete actions — "Detect breaking changes", "check whether the committed spec is stale", "run the OAS checks, interpret findings, and guide fixes" — all tied to a named file. Not a 5 because it omits a capability the body covers prominently (regenerating the spec), leaving a minor coverage gap.	4 / 5
Completeness	The "what" is clearly and explicitly stated (detect breaking changes, check staleness, run/interpret/guide fixes), but there is no "Use when..." clause or equivalent trigger guidance — the "when" is only weakly implied by naming the spec file. Per the rubric, a missing explicit 'when' caps completeness at 3.	3 / 5
Trigger Term Quality	Good natural terms users would say: "breaking changes", "openapi.yaml", "stale", "OAS", "spec", "findings". Not a 5 because common synonyms/extensions like "swagger", "API spec", or "diff the API" are missing.	4 / 5
Distinctiveness Conflict Risk	Clear niche — OpenAPI spec maintenance for a specific file (docs/mapi/openapi.yaml) with OAS-specific vocabulary — making it unlikely to trigger for unrelated skills; distinct triggers like "breaking changes" and "spec is stale" are unambiguous.	5 / 5
	Total	16 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
referenced_paths_exist	Referenced path issues: 6 missing	Warning

	Total	15 / 16 Passed

Repository: gravitee-io/gravitee-access-management
Path: docs/agent-standards/skills/check-oas/SKILL.md
Commit: 6bfca2c

Reviewed: 5 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.