fake-model-faults

Make the desktop's model provider fail on demand (connection refused, DNS not found, socket reset, 429, 5xx, 401, 402, stalled or cut stream) with a local fault server and test providers. Use when reproducing or designing model error and retry states without a real outage.

Quality

94%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

High

Do not use without reviewing

Quality

Content

92%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is a model of lean, command-first skill writing: a tight setup, a table mapping each test model to its error, and a symmetric cleanup, all backed by real bundle scripts. The only weakness is the unresolved CDP port placeholders, which keep the commands from being fully copy-paste ready.

Suggestions

Show how to obtain the CDP port (e.g., a command or pointer into the drive-desktop-cdp skill) so the `CDP_URL=http://127.0.0.1:<cdp>` line is copy-paste ready.

Repeat the concrete CDP_URL example in the Clean up section instead of the bare `CDP_URL=…` placeholder, so cleanup works without scrolling back to Set up.

Add a quick verification after cleanup, e.g. confirm the fault-lab DELETE returns 200 before killing the fault server, to close the workflow loop.

Dimension	Reasoning	Score
Conciseness	The ~40-line body contains only commands, a mapping table, and one justified "why" sentence (CDP throttling cannot produce engine errors). There is no concept explanation or padding to trim, matching the lean anchor 5.	5 / 5
Actionability	Commands are copy-paste ready with real script paths, port, env vars, and an expected result ("expect 200"), but two placeholders are never resolved: "<cdp>" in Set up and "CDP_URL=…" in Clean up, with no hint at how to obtain the CDP port. This is a minor gap, matching anchor 4 rather than fully-executable anchor 5.	4 / 5
Workflow Clarity	A clear Set up → use → Clean up sequence with an explicit validation checkpoint ("refresh-providers ... expect 200"), a preventive gotcha (writing the engine config directly "does not stick"), a safety step ("backs up runtime.sqlite"), and restore steps in cleanup. For this simple, single-purpose skill the sequence is unambiguous, matching anchor 5.	5 / 5
Progressive Disclosure	Under 50 lines with three clean sections and no need for separate reference files; both bundle scripts referenced in the body (scripts/fault-server.mjs, scripts/fault-providers.py) exist and match the documented port and subcommands. Per the simple-skill guideline, well-organized sections alone merit the top anchor.	5 / 5
	Total	19 / 20 Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is exemplary: it names the exact failure modes it can produce, states the mechanism, and gives an explicit "Use when..." trigger clause. Only minor synonym coverage for trigger phrasing keeps trigger_term_quality at 4.

Dimension	Reasoning	Score
Specificity	"fail on demand (connection refused, DNS not found, socket reset, 429, 5xx, 401, 402, stalled or cut stream) with a local fault server and test providers" lists multiple specific concrete behaviors plus the mechanism that produces them. Coverage of failure modes is comprehensive rather than having minor gaps, matching the anchor 5.	5 / 5
Completeness	The "what" is explicit and concrete (enumerated fault types delivered via a local fault server and test providers) and the "when" is an explicit trigger clause ("Use when reproducing or designing model error and retry states without a real outage"). Both are answered clearly, matching anchor 5.	5 / 5
Trigger Term Quality	Natural phrases users would say are present ("fail on demand", "model error and retry states") alongside raw error tokens (429, 5xx, connection refused, DNS not found). A few common variations are missing (e.g., "simulate API errors", "test error handling", "timeout"), so it fits anchor 4 rather than comprehensive anchor 5.	4 / 5
Distinctiveness Conflict Risk	The niche is narrow and well-defined — deliberately making the model provider fail for error/retry testing — with distinct trigger language, so conflict with other skills is minimal. Anchor 5.	5 / 5
	Total	19 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: different-ai/openwork
Path: .opencode/skills/fake-model-faults/SKILL.md
Commit: e026c3d

Reviewed: 3 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.