postman2pytest

Convert a Postman Collection v2.1 JSON file into a runnable pytest test suite using the postman2pytest CLI. Use when the user has a Postman collection (a .postman_collection.json or v2.1 JSON export) and wants to run it as pytest in CI, when migrating from Postman/Newman to a Python-native test stack, when bridging Postman-documented APIs into a pytest-based regression suite, when the user asks to generate pytest tests from Postman, or when the user mentions wanting to keep Postman as the source of truth but run the suite with pytest.

1.00x

Quality

—

Does it follow best practices?

Impact

100%

1.00x

Average score across 2 eval scenarios

Securityby

Passed

No known issues

Quality

Content

92%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A tight, highly actionable body with a well-sequenced validated workflow. Its one real defect is progressive disclosure: the body repeatedly points to a REFERENCE.md that does not exist in the bundle, while the bundle's actual script and asset files go unreferenced.

Suggestions

Add the missing REFERENCE.md to the bundle (or change the body references to point at files that actually exist) so the signposted detail is reachable.

Mention the existing bundle files in the body — e.g. reference scripts/generate_stress_collection.py from the "Testing at scale" / stress-test discussion and assets/demo.gif where appropriate — so the bundle is discoverable.

Verify each inline path (REFERENCE.md sections "CI integration", "Testing at scale", error tree) resolves to real anchored content rather than dangling section names.

Dimension	Reasoning	Score
Conciseness	The body is lean and assumes competence: it never explains what Postman, pytest, or CI are, and every section (Quick start, Example, Common errors, CI) earns its tokens rather than padding with background.	3 / 3
Actionability	Guidance is copy-paste ready: "pip install postman2pytest", "postman2pytest --collection my_api.postman_collection.json --out tests/test_api.py", "grep -c '^def test_' tests/test_api.py", and "pytest tests/test_api.py -v" are all fully executable commands.	3 / 3
Workflow Clarity	The 7-step Quick start is clearly sequenced with an explicit validation checkpoint (step 4: sanity-check that the test count matches the collection's request count) and a feedback loop (step 7: re-run step 3 whenever the collection changes).	3 / 3
Progressive Disclosure	References are clearly signaled and intended as one-level-deep ("Full CLI reference... live in REFERENCE.md next to this file"), but REFERENCE.md is absent from the bundle and the actual bundle files (scripts/generate_stress_collection.py, assets/demo.gif) are never referenced, so the navigation is effectively broken.	2 / 3
	Total	11 / 12 Passed

Description

90%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, third-person description that clearly states the capability and provides rich explicit "Use when" triggers with good natural-term coverage. Its only weakness is that it describes one conversion action rather than a breadth of distinct capabilities.

Dimension	Reasoning	Score
Specificity	"Convert a Postman Collection v2.1 JSON file into a runnable pytest test suite using the postman2pytest CLI" names the input format, tool, and output concretely, but lists essentially a single conversion action rather than the multiple distinct actions the top anchor calls for.	2 / 3
Completeness	It explicitly answers what (convert v2.1 JSON into a runnable pytest suite via the CLI) and when (a multi-clause "Use when the user has... / wants to run it as pytest in CI / migrating... / asks to generate pytest tests from Postman") with explicit triggers.	3 / 3
Trigger Term Quality	Phrases like "Postman collection", ".postman_collection.json", "generate pytest tests from Postman", "run it as pytest in CI", and "migrating from Postman/Newman" are natural terms users would actually say, with good variation coverage.	3 / 3
Distinctiveness Conflict Risk	The Postman-to-pytest conversion niche is narrow with distinct, specific triggers, making conflict with other skills unlikely.	3 / 3
	Total	11 / 12 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: golikovichev/postman2pytest
Commit: 3b02ce8

Reviewed: 2 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.