CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/schemathesis-fuzzing

Generates property-based API tests automatically from an OpenAPI 2/3.x or GraphQL schema using Schemathesis, runs them via the `schemathesis run` CLI or as a pytest decorator, configures the canonical checks (status_code_conformance, response_schema_conformance, content_type_conformance, response_headers_conformance, not_a_server_error), and gates CI on schema-conformance failures plus 5xx detection. Use when the project ships an OpenAPI or GraphQL schema and the team wants schema-driven coverage that scales as the API evolves.

72

Quality

91%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

86%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable skill body: copy-paste CLI and pytest examples, flag and check tables, anti-patterns with fixes, and a clean one-level-deep push of CI/hook detail into a real reference file. The only improvements are trimming a few redundant sentences and making the failure-triage loop explicit in the workflow steps.

Suggestions

Conciseness: fold the Install section into 'How to use' step 2 (the pip/uvx commands are already stated there) and drop the 'uvx is the uv runner' aside and repeated 'Per [schemathesis-readme]' attributions.

Workflow clarity: turn step 3's failure handling into an explicit loop in 'How to use' — 'if a check fails: reproduce with the printed curl command + Hypothesis seed, triage per the anti-patterns table, and only then promote to pytest/CI' — so the validation feedback cycle is one visible sequence.

DimensionReasoningScore

Conciseness

The body is efficient — dense flag/check tables, a worked example, anti-patterns, and limitations, with almost no explanation of concepts Claude already knows. Minor instances of over-explanation could be trimmed: 'uvx is the `uv` runner for one-off tool execution', a separate Install section repeating 'pip install schemathesis' already stated in How-to-use, and per-section 'Per [schemathesis-readme]' attribution. This fits the 4 anchor ('efficient; minor instances of over-explanation') rather than the 5 anchor's 'every token earns its place', and is well above 3's 'some unnecessary explanation'.

4 / 5

Actionability

Guidance is fully executable: 'pip install schemathesis', 'uvx schemathesis run <schema-url>', a copy-paste multi-flag CLI worked example, and a complete runnable pytest block (`schema = schemathesis.openapi.from_url(...)`, `@schema.parametrize()`, `case.call_and_validate()`). The common cases (CLI run, restricted checks, pytest promotion, CI gating via the reference) are all covered with real commands — a match for the 5 anchor.

5 / 5

Workflow Clarity

The 'How to use' section gives a clear 4-step sequence (confirm schema → install → run against staging → promote to pytest + CI gate) and step 3 embeds a verification checkpoint ('reproduce any failure from the printed curl command + Hypothesis seed'), with the anti-patterns table adding triage rules for failures. It falls short of the 5 anchor because the failure-handling loop (reproduce → triage → fix or file) is distributed across sections rather than presented as an explicit validate/fix/retry sequence in the workflow itself — matching the 4 anchor's 'clear sequence with most checkpoints present; minor validation gaps'.

4 / 5

Progressive Disclosure

The body is a genuine overview and the deep material (GitHub Actions YAML, before_call hooks, cadence table) lives one level deep in references/ci-and-pytest-integration.md, which exists and is well-signaled at exactly the three points it is needed (auth hooks, CI wiring, cadence). Section structure (Overview / When to use / How to use / CLI / checks / pytest / CI / anti-patterns / limitations / references) makes navigation easy — a match for the 5 anchor; the slight summary-vs-detail overlap on cadence numbers is the standard progressive-disclosure pattern, not a 4-level organization gap.

5 / 5

Total

18

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it enumerates concrete capabilities, names the tool and schema formats, and includes an explicit 'Use when...' trigger clause. The only gap is a few natural trigger synonyms (swagger, API fuzzing) and file extensions.

DimensionReasoningScore

Specificity

The description lists multiple concrete actions — 'Generates property-based API tests automatically from an OpenAPI 2/3.x or GraphQL schema', 'runs them via the `schemathesis run` CLI or as a pytest decorator', 'configures the canonical checks (status_code_conformance, response_schema_conformance, ...)', and 'gates CI on schema-conformance failures plus 5xx detection' — with comprehensive coverage of what the skill does. This matches the anchor for multiple specific concrete actions; nothing in it is vague or padded, so a 4 (minor gaps) would undersell the coverage.

5 / 5

Completeness

It explicitly answers both what (generate, run, configure checks, gate CI) and when with a concrete trigger clause: 'Use when the project ships an OpenAPI or GraphQL schema and the team wants schema-driven coverage that scales as the API evolves.' This is a direct match for the 5 anchor ('clearly and explicitly answers both what AND when with concrete trigger phrases'); a 4 would require the 'when' to be less explicit than it is.

5 / 5

Trigger Term Quality

Good keyword coverage with natural terms users would say: 'OpenAPI', 'GraphQL schema', 'API tests', 'Schemathesis', '5xx', 'CI', 'schema-driven coverage'. A few common natural variants are missing — 'swagger' (the most common synonym for OpenAPI), 'API fuzzing', and file extensions like '.yaml'/'openapi.json' — which keeps it below the 5 anchor's 'comprehensive coverage of natural terms including synonyms and file extensions', but it is clearly above the 3 anchor's 'missing common variations or synonyms'.

4 / 5

Distinctiveness Conflict Risk

It names a specific tool ('using Schemathesis'), specific schema formats (OpenAPI 2/3.x, GraphQL), and a specific failure model (schema-conformance plus 5xx detection) — a clear niche with distinct triggers and minimal conflict risk with example-based testing skills. It fits the 5 anchor better than the 4 anchor's 'minor overlap risk with closely related skills'.

5 / 5

Total

19

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

Table of Contents