run-documentation-examples

Extract Python examples from markdown docs and run them (including LLM examples). Use when validating documentation, after doc changes, or to verify all doc examples execute correctly.

1.51x

Quality

82%

Does it follow best practices?

Impact

100%

1.51x

Average score across 3 eval scenarios

Securityby

Passed

No known issues

Quality

Content

64%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

This is a solid, actionable skill that provides concrete executable commands for multiple approaches to running documentation examples. Its main weaknesses are the lack of explicit validation/feedback loops in the workflow (especially important since this involves batch execution of extracted code) and the absence of bundle files despite referencing specific script paths. The content could be slightly tighter by trimming the 'When to Use' section and consolidating the multiple workflow options more clearly.

Suggestions

Add an explicit validation/feedback loop to the workflow: after running examples, specify what success looks like and what to do on failure (e.g., 'If pytest shows failures: 1. Check the specific example in the markdown source 2. Fix the example 3. Re-run only failed tests with pytest --lf').

Include the referenced scripts (run_documentation_examples.py, extract_documentation_examples.py) as bundle files, or remove the specific path references if they're not available.

Consolidate the three workflow options (pytest, CLI, manual) into a clear primary recommendation with alternatives briefly noted, rather than presenting them as near-equal options.

Dimension	Reasoning	Score
Conciseness	Mostly efficient but has some unnecessary explanation (e.g., 'LLM examples run as-is—they make real API calls' is somewhat obvious given context, and the 'When to Use' section states things Claude could infer). The prerequisites section is useful but slightly verbose.	2 / 3
Actionability	Provides fully executable, copy-paste-ready commands for multiple approaches (pytest, CLI, manual two-step). Specific commands for prerequisites (ollama pull, pip install extras) and environment setup are concrete and actionable.	3 / 3
Workflow Clarity	Multiple workflow options are presented clearly with specific commands, and the failure handling with resume capability is a nice touch. However, there's no explicit validation checkpoint or feedback loop—after running examples, there's no 'check output for X' or 'if failures occur, do Y before Z' structured sequence. The failure handling section describes behavior but doesn't provide a clear error recovery workflow.	2 / 3
Progressive Disclosure	Content is reasonably organized with clear section headers, but no bundle files are provided despite references to specific paths like `.cursor/skills/run-documentation-examples/scripts/` and `scripts/extract_documentation_examples.py`. The skill is somewhat monolithic—the prerequisites and multiple workflow alternatives could benefit from better separation or at least clearer signaling of primary vs. alternative paths.	2 / 3
	Total	9 / 12 Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

This is a strong description that clearly communicates a specific, well-defined capability. It uses third person voice, lists concrete actions, includes natural trigger terms, and explicitly states both what the skill does and when to use it. The niche is distinct enough to avoid conflicts with other skills.

Dimension	Reasoning	Score
Specificity	Lists multiple specific concrete actions: 'Extract Python examples from markdown docs', 'run them (including LLM examples)', and mentions validating documentation and verifying doc examples execute correctly.	3 / 3
Completeness	Clearly answers both what ('Extract Python examples from markdown docs and run them') and when ('Use when validating documentation, after doc changes, or to verify all doc examples execute correctly') with explicit trigger guidance.	3 / 3
Trigger Term Quality	Includes natural keywords users would say: 'Python examples', 'markdown docs', 'documentation', 'doc changes', 'doc examples', 'execute'. These cover common variations of how a user would describe this task.	3 / 3
Distinctiveness Conflict Risk	Very specific niche: extracting and running Python code examples from markdown documentation. This is unlikely to conflict with general Python execution skills or general documentation skills due to the precise combination of extraction + execution + markdown docs.	3 / 3
	Total	12 / 12 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 11 / 11 Passed

Validation for skill structure

No warnings or errors.

Repository: sandialabs/talkpipe
Commit: 0cf1686

Reviewed: 17 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.