CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

launchdarkly-experiment-setup

Set up and run experiments in LaunchDarkly. Create experiments with metrics, treatments, and flag config, start iterations to collect data, swap design between iterations, and stop with a winner.

Quality

81%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

77%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A highly actionable, well-sequenced operational guide with executable JSON examples, explicit approval gates, and strong validation loops. Its weaknesses are redundancy (the start-gating and approval rules repeated many times, plus a partly duplicated tool listing and Edge Cases table) and a monolithic single-file structure where a tool reference or edge-case reference file would reduce token load.

Suggestions

State the start-gating and approval rules once in a single authoritative section and reference it from Steps 4 and 6, the lifecycle, and the Edge Cases table instead of restating them; this would also fix the duplicate 'get-experiments' entry under both Required and Optional MCP tools.

Move the Optional MCP tools catalog and the Edge Cases table into a references/ file (e.g. references/tools.md, references/edge-cases.md) linked from SKILL.md to cut context-window cost.

Consolidate the statistics/methodology default, which currently appears three times (Before you create anything, Step 3 twice), into one location.

DimensionReasoningScore

Conciseness

The content is dense and operational rather than padded with concepts Claude already knows, but the start-gating rule is restated roughly six times (intro, lifecycle, Core Principles, Step 4, Step 6, What NOT to Do, Edge Cases), the statistics default appears three times, 'get-experiments' is listed under both Required and Optional MCP tools, and the Edge Cases table largely duplicates earlier rules. Not 2 because there is no generic-concept explanation; not 4 because the redundancy exceeds minor trimming.

3 / 5

Actionability

Copy-paste-ready JSON examples cover all four critical calls — create-experiment, start-experiment-iteration, save-and-start-experiment-iteration, and stop-experiment-iteration — alongside exact field names (primarySingleMetricKey, winningTreatmentId, flagConfigVersion) and concrete heuristics for suggesting control variations and randomization units. Not 4 because the common cases are fully covered, not just mostly.

5 / 5

Workflow Clarity

The gather → design → 'STOP HERE' approval → create → start → verify → evolve → stop sequence is explicit, with validation checkpoints (Step 5's currentIteration.status === "running" checks, event-arrival warnings before starting) and error-recovery loops (re-show design on change, re-run get-metric-events on corrected event key, handle update-experiment 'skipped' fields). Not 4 because feedback loops and checkpoints are explicit throughout, not just mostly present.

5 / 5

Progressive Disclosure

Section structure with headers is clear, but this is a single 330-line file with no bundle files; the 15-entry MCP tool catalog and the 16-row Edge Cases table are content that could be split into reference files. Not 2 because the structure is well-organized with clear headers rather than minimal; not 4 because substantial reference-worthy content is inlined in SKILL.md.

3 / 5

Total

16

/

20

Passed

Description

75%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A specific, well-scoped description that concretely covers the experiment lifecycle from creation to stopping with a winner. Its main weakness is the absence of any explicit 'when to use' trigger guidance and of common synonyms like 'A/B testing' or 'feature flags' that users might naturally say.

Suggestions

Add an explicit trigger clause, e.g. 'Use when setting up A/B tests or experiments in LaunchDarkly, creating experiment metrics or treatments, or deciding when to stop an iteration and declare a winner.'

Include natural synonyms users would say, such as 'A/B test', 'feature flag experiment', or 'split test', to improve trigger-term coverage.

DimensionReasoningScore

Specificity

Lists multiple concrete actions spanning the full lifecycle — 'Create experiments with metrics, treatments, and flag config', 'start iterations to collect data', 'swap design between iterations', and 'stop with a winner' — with no significant coverage gaps. A 4 would require missing actions; the create/start/evolve/stop arc is comprehensive for the domain.

5 / 5

Completeness

The 'what' is clear and concrete (create, start, swap, stop experiments in LaunchDarkly), but there is no 'Use when...' clause or equivalent explicit trigger guidance, which caps completeness at 3 per the judging guidelines. Not 2 because the 'what' is strong and specific rather than vague.

3 / 5

Trigger Term Quality

'experiments' and 'LaunchDarkly' are natural terms users would say, giving good keyword coverage. Not 5 because common synonyms like 'A/B test', 'feature flag', or 'split test' are absent; not 3 because the core terms are well-chosen and directly match user phrasing.

4 / 5

Distinctiveness Conflict Risk

Naming LaunchDarkly plus the experiment lifecycle (metrics, treatments, iterations, flag config) carves out a clear niche with distinct triggers and minimal overlap risk with other skills. Not 4 because no closely related skill domain would plausibly collide with these terms.

5 / 5

Total

17

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

launchdarkly/ai-tooling

Path

skills/experiments/launchdarkly-experiment-setup/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.