CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

experiments

Use when adding, changing, or reviewing a VS Code A/B experiment, either an experiment-controlled setting or a treatment read with the assignment service, and when deciding whether and where to log an `experimentTrigger` event for a triggered ExP scorecard.

72

Quality

90%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

100%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

An exemplary SKILL.md body: every sentence adds non-obvious, repo-specific knowledge (APIs, event names, settings keys, arm-parity pitfalls) with zero padding, concrete copy-paste-ready code, a clear decision workflow with an explicit validation step, and clean single-file organization with one well-signaled source reference. No dimension shows a gap worth penalizing.

DimensionReasoningScore

Conciseness

Lean and dense throughout: every token carries repo-specific knowledge Claude cannot infer — 'experiment: { mode: 'auto' }', 'IWorkbenchAssignmentService.getTreatment<T>(name)', 'experiments.override.<treatment name>', 'monacoworkbench/experimentTrigger', the once-per-window semantics, the agent-host forwarding caveat. No general concepts (what A/B testing is, how telemetry works) are re-explained, and the one rationale sentence ('Dilution only costs statistical power, while a biased trigger gives wrong results') directly justifies the decision rules that follow, so it earns its place.

5 / 5

Actionability

Fully concrete, executable guidance: exact schema properties to add, exact API calls with generic types, an exact user-settings override string, named helper functions with copy-paste-ready call signatures ('logSettingExperimentTrigger(telemetryService, MY_SETTING_ID)'), a correctly-ordered code example, named test service 'TestExperimentTriggerTelemetryService', and named code anchors (experimentTrigger.ts, AgentHostGitStateService, chat.agentMerge.enabled). This matches the level-5 anchor for an instruction skill with concrete, specific guidance throughout.

5 / 5

Workflow Clarity

The processes are clearly sequenced with explicit checkpoints: choose the control mechanism (setting vs. treatment, with the decision criterion 'users could reasonably choose the behavior themselves'), decide whether a trigger applies (concrete in/out cases), fire it with a decision table plus an annotated code example showing arm-independent ordering, then validate — 'Test each trigger in every arm with TestExperimentTriggerTelemetryService'. The pitfalls list ('Find every place where the setting or treatment changes behavior') acts as a checklist, and 'start a new experiment iteration' after moving a trigger is corrective guidance, matching the level-5 anchor.

5 / 5

Progressive Disclosure

The skill is self-contained at ~63 lines with no bundle files, well-organized under clear sections ('Controlling behavior', 'Trigger events', 'When to use one', 'When to fire it'), and its single reference — the source-file link to experimentTrigger.ts — is well-signaled and one level deep. Nothing is inlined that clearly belongs in a separate file at this scale, so navigation is easy and content placement is appropriate, matching the level-5 anchor.

5 / 5

Total

20

/

20

Passed

Description

78%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, highly targeted description that names concrete actions, artifacts, and mechanisms with an explicit, multi-clause 'Use when' trigger. Its only weakness is that the skill's purpose is folded entirely into the trigger clause rather than stated as a separate 'what' statement, and a few natural synonyms are absent. Distinctiveness is excellent — there is essentially no conflict risk with other skills.

Suggestions

Open with a short capability statement (e.g., 'Guides implementation of VS Code A/B experiments…') before the 'Use when' clause so the 'what' is stated explicitly, not just implied by the trigger enumeration.

Add one or two natural synonyms users might say, such as 'feature flag' or 'flight', to broaden trigger-term coverage.

DimensionReasoningScore

Specificity

Names several concrete actions in a specific domain — 'adding, changing, or reviewing a VS Code A/B experiment', 'an experiment-controlled setting', 'a treatment read with the assignment service', 'log an `experimentTrigger` event for a triggered ExP scorecard' — going beyond the 1–2 actions of the level-3 anchor. Not a 5 because coverage has minor gaps: the skill also guides trigger placement and testing, which the description does not hint at.

4 / 5

Completeness

The 'when' is explicit and concrete ('Use when adding, changing, or reviewing a VS Code A/B experiment… and when deciding whether and where to log an `experimentTrigger` event'), satisfying the Use-when requirement. The 'what' is conveyed only through the enumerated tasks rather than stated as its own capability statement, so it does not clearly match the level-5 anchor where both are explicit; it is well above the level-3 anchor where 'when' is missing or weakly implied.

4 / 5

Trigger Term Quality

Good natural keyword coverage — 'experiment', 'A/B experiment', 'treatment', 'scorecard', 'experimentTrigger', 'VS Code' — phrases a repo contributor would actually say. A few natural terms are missing (e.g., 'feature flag', 'flight', 'telemetry trigger'), so it falls just short of the comprehensive-synonym coverage of the level-5 anchor.

4 / 5

Distinctiveness Conflict Risk

A clear niche with distinct triggers — VS Code A/B experiments, ExP scorecards, the experimentTrigger event — that no generically-described skill would overlap with. This matches the level-5 anchor ('clear niche with distinct triggers; minimal conflict risk').

5 / 5

Total

17

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

relative_links

Relative link issues: 1 suspicious

Warning

Total

15

/

16

Passed

Repository

microsoft/vscode

Path

.github/skills/experiments/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.