creating-experiments

Guides agents through the 3-step experiment creation flow: defining the hypothesis, configuring rollout, and setting up analytics. Delegates rollout decisions to configuring-experiment-rollout and metric setup to configuring-experiment-analytics. TRIGGER when: user asks to create a new experiment or A/B test, OR when you are about to call experiment-create. DO NOT TRIGGER when: user is updating an existing experiment, managing lifecycle, or only browsing experiments.

Quality

90%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

93%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is an efficient, highly actionable walkthrough with a complete executable payload and clean delegation to sibling skills. The only gap is the absence of an explicit validation/error-recovery loop, which matters little for a non-destructive creation flow.

Dimension	Reasoning	Score
Conciseness	Lean and efficient throughout — it assumes Claude's competence, adds only PostHog-specific knowledge (the "web" toolbar caveat, deprecated `parameters` keys), and avoids explaining basic A/B-test or MCP concepts.	5 / 5
Actionability	Provides a copy-paste-ready, complete `experiment-create` JSON payload covering the common 50/50 case, plus specific documented keys and conventions for variants and rollout percentages.	5 / 5
Workflow Clarity	The 3-step flow is clearly sequenced with delegation checkpoints ("Before asking any rollout question, load configuring-experiment-rollout"), defaults, and a post-creation checklist; not a 5 because there is no explicit error-recovery loop, though the operation is non-destructive.	4 / 5
Progressive Disclosure	Well-organized into clear sections with detail appropriately offloaded to one-level-deep sibling skills (configuring-experiment-rollout, configuring-experiment-analytics, managing-experiment-lifecycle); no bundle files are needed or missing.	5 / 5
	Total	19 / 20 Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is strong: it states concrete actions, gives explicit positive and negative trigger guidance, and draws clear boundaries against sibling experiment skills. It falls just short of perfect specificity and trigger-term breadth.

Dimension	Reasoning	Score
Specificity	Names the domain and three concrete actions ("defining the hypothesis, configuring rollout, and setting up analytics") plus the delegated sibling skills, with only minor coverage gaps; not a full 5 because the actions are described at a high level rather than enumerated exhaustively.	4 / 5
Completeness	Explicitly answers both what (the "3-step experiment creation flow") and when ("TRIGGER when: user asks to create a new experiment or A/B test, OR when you are about to call experiment-create") with concrete trigger phrases, reinforced by explicit negative triggers.	5 / 5
Trigger Term Quality	Includes natural phrases users would say ("create a new experiment", "A/B test") plus the tool trigger ("experiment-create"); a few natural synonyms are missing, keeping it just below a 5.	4 / 5
Distinctiveness Conflict Risk	Occupies a clear niche (creation) with explicit "DO NOT TRIGGER when" boundaries for updating, lifecycle, and browsing, plus delegation to sibling skills, giving minimal overlap risk.	5 / 5
	Total	18 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: PostHog/posthog
Path: products/experiments/skills/creating-experiments/SKILL.md
Commit: 5b94e47

Reviewed: about 11 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.