generate-synthetic-data

Create diverse synthetic test inputs for LLM pipeline evaluation using dimension-based tuple generation. Use when bootstrapping an eval dataset, when real user data is sparse, or when stress-testing specific failure hypotheses. Do NOT use when you already have 100+ representative real traces (use stratified sampling instead), or when the task is collecting production logs.

Quality

73%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./skills/generate-synthetic-data/SKILL.md

Quality

Content

71%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, efficiently written skill with a clear multi-step workflow and explicit human-in-the-loop checkpoints. The main weakness is actionability: several templates remain abstract placeholders rather than concrete, copy-pasteable examples.

Suggestions

Replace the bracketed placeholder templates in Steps 1–2 with at least one fully filled-in concrete example (beyond the real-estate illustration) so the structure is immediately executable.

Add a short validate-and-retry feedback loop after Step 6 (e.g. inspect traces for new failure categories, regenerate to fill gaps) to raise workflow clarity for a batch operation.

Tighten the Anti-Patterns section by stating each anti-pattern as a one-line 'Don't: ... because ...' to cut redundant rationale.

Dimension	Reasoning	Score
Conciseness	Mostly lean prose that assumes Claude's competence and avoids explaining basic concepts; a few framing sentences ('The user's domain knowledge is essential here') and repeated rationales in Anti-Patterns could be trimmed.	4 / 5
Actionability	Provides ready-to-use prompt templates for Steps 3 and 4, but Step 1/Step 2 templates are placeholders (bracketed '[Name]', '[value_a]') and there is no executable command or concrete runnable example a user can copy directly.	3 / 5
Workflow Clarity	A clear six-step sequence with explicit review checkpoints (user confirms tuples in Step 2, quality filter in Step 5, saturation heuristic in Step 6); minor gap is the absence of an explicit validate-and-retry loop after running queries through the pipeline.	4 / 5
Progressive Disclosure	Well-organized into concise overview sections with no bundle files needed; content is appropriately inlined for a self-contained skill and easy to navigate via clear section headers.	5 / 5
	Total	16 / 20 Passed

Description

76%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, well-scoped description that clearly states what the skill does and when to use it, with helpful negative triggers reducing conflict risk. Trigger-term naturalness is the weakest dimension: it leans technical and omits everyday synonyms users might say.

Suggestions

Add common-language trigger synonyms like 'fake test data', 'mock inputs', or 'made-up examples' so the skill surfaces for users who don't say 'synthetic' or 'eval dataset'.

Consider a brief mention of the output form (e.g. 'test queries/traces') to sharpen specificity beyond the generation method.

Dimension	Reasoning	Score
Specificity	Lists several concrete actions ('Create diverse synthetic test inputs', 'dimension-based tuple generation', 'stress-testing specific failure hypotheses'), naming the domain and multiple specific capabilities with only minor coverage gaps around output format.	4 / 5
Completeness	Explicitly answers both 'what' (create synthetic test inputs via dimension-based tuple generation) and 'when' ('Use when bootstrapping an eval dataset, when real user data is sparse, or when stress-testing specific failure hypotheses'), plus explicit negative triggers ('Do NOT use when...').	5 / 5
Trigger Term Quality	Includes relevant terms ('synthetic test inputs', 'eval dataset', 'stress-testing failure hypotheses') but lacks the everyday synonyms a user would naturally say (e.g. 'fake test data', 'mock inputs') and file-type/extension cues.	3 / 5
Distinctiveness Conflict Risk	The dimension-based tuple-generation niche and explicit 'Do NOT use' boundary guidance make it mostly distinct from generic eval/data skills, with only minor overlap risk against stratified-sampling or data-collection skills.	4 / 5
	Total	16 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: hamelsmu/evals-skills
Path: skills/generate-synthetic-data/SKILL.md
Commit: 22418da

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.