Tessl • The package manager for agent skills

SkillPublic

signals-scout-experiments

Signals scout for PostHog A/B experiments. Watches running experiments for validity threats — sample ratio mismatch, contamination, exposure stalls, mid-run flag mutations — and lifecycle drift.

Quality

78%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

—

The risk profile of this skill

Fix and improve this skill with Tessl

tessl review fix ./products/signals/skills/signals-scout-experiments/SKILL.md

Quality

Content

81%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A highly actionable, expertly sequenced scout skill with executable tool calls, SQL, and rigorous validation gates; its main weakness is density — the body is long and could trim repeated narrative framing, and it carries inlined reference material that progressive disclosure would normally move to bundle files.

Suggestions

Tighten the repeated narrative framing (e.g. the 'config-vs-data contradiction' paragraphs recur) to reduce token load without losing the signal-vs-noise principle.

Consider moving the MCP tools catalog and the SQL snippets into a references/ file so SKILL.md stays an overview, improving progressive disclosure from 4 toward 5.

The skill is already strong on validation; no change needed to the disqualifiers or sample-size gates.

Dimension	Reasoning	Score
Conciseness	Mostly efficient and expert-voiced with no basic-concept padding, but the 260-line body is dense with extended narrative framing and repeated caveats that could be tightened without losing clarity.	3 / 5
Actionability	Fully executable: concrete MCP tool calls with parameters, two copy-paste-ready SQL blocks, exact field paths, and quantified sample-size gates (>3σ, n<1000 thresholds).	5 / 5
Workflow Clarity	Clear sequenced run flow (orient → profile → explore → decide → close out) with explicit validation checkpoints, sample-size gating, and a disqualifiers checklist that feeds the edit/author/remember/skip decision.	5 / 5
Progressive Disclosure	Well-structured with clear section headers and an explicit note that the report-channel contract lives in the harness prompt rather than here, but no bundle files exist so all referenceable detail (tool catalog, SQL) is inlined rather than split into separate files.	4 / 5
	Total	17 / 20 Passed

Description

67%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A specific, well-scoped third-person description that names concrete actions and a clear niche, but it omits any explicit 'when to use' trigger clause, capping completeness and leaving natural trigger terms thin.

Suggestions

Append a 'Use when...' clause naming natural user triggers, e.g. 'Use when checking running PostHog A/B experiments for validity threats (sample ratio mismatch, contamination, stalled exposures, mid-run flag edits).'

Add common synonyms like 'A/B test' and 'experiment results' so the description matches phrases a user would naturally say.

Keep the third-person voice — it is correct here — while adding the missing trigger guidance.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions (watches SRM, contamination, exposure stalls, mid-run flag mutations, lifecycle drift) with comprehensive coverage of the experiment-validity domain.	5 / 5
Completeness	Has a clear, concrete 'what' but entirely missing any 'when' / 'Use when...' trigger clause, which the rubric caps completeness at 3.	3 / 5
Trigger Term Quality	Names 'A/B experiments' and 'PostHog' but lacks the common synonyms a user would naturally say ('experiment validity', 'A/B test', 'experiment results'); no file extensions or phrasings like 'experiments' alone.	3 / 5
Distinctiveness Conflict Risk	Occupies a clear niche (PostHog experiment validity scouting) with distinct triggers unlikely to fire for unrelated skills.	5 / 5
	Total	16 / 20 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
metadata_version	'metadata.version' is missing	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	14 / 16 Passed

Repository: PostHog/posthog
Path: products/signals/skills/signals-scout-experiments/SKILL.md
Revision: 130f3a1

Reviewed: 29 days ago

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.

signals-scout-experiments

Table of Contents

Is this your skill?