creating-replay-vision-scanners

Guides agents through creating and safely sizing a Replay Vision scanner: choosing the scanner type (monitor/classifier/scorer/summarizer), shaping the RecordingsQuery that selects sessions, and — crucially — estimating the credits it will spend and checking the org's remaining budget before creating, so a broad scanner doesn't exhaust the budget on its first scheduled sweep. TRIGGER when: user asks to create, set up, or configure a Replay Vision scanner, OR when you are about to call vision-scanners-create, OR when widening an existing scanner's query, sampling_rate, or sampling_mode (or moving it to a pricier model) via vision-scanners-update. DO NOT TRIGGER when: only reading scanners or observations, deleting a scanner, or running an existing scanner against a single session on demand (vision-scanners-scan-session). For a one-off question about sessions you already have, use vision-scanners-inline-scan-create rather than creating a scanner — the skill's first section covers when that applies.

Quality

91%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Medium

Suggest reviewing before use

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A high-quality procedural skill body: actionable tool calls and a ready-to-use create example, an explicit pre-creation budget gut-check with feedback loops, and clean sectioning. It is only slightly verbose in a few rationale passages and is self-contained with no bundle files to disclose.

Suggestions

Tighten the opening two paragraphs and the 'Confirmation here is a conversation step, not an API capability' paragraph to lift conciseness toward 5.

If the per-type scanner_config table or the model pricing list grows, move it to a references/ file and link from the body to give progressive_disclosure a clear one-level-deep reference and room to score 5.

Dimension	Reasoning	Score
Conciseness	Dense and focused on product-specific knowledge Claude would not already know (scanner schedules, credits, sampling levers); it does not pad with general concepts. A few prose-heavy rationale passages (the opening two paragraphs, the 'Confirmation here is a conversation step' paragraph) could be trimmed, so it sits at 4 rather than 5.	4 / 5
Actionability	Names exact tools (vision-scanners-estimate-create, vision-quota-retrieve, vision-scanners-create), enumerates their return fields, gives a copy-paste-ready JSON create example, and provides an explicit budget-proration formula block — fully executable guidance covering the common cases.	5 / 5
Workflow Clarity	A clearly sequenced four-step flow with an explicit validation checkpoint ('Step 3: Size it — the gut-check (do not skip)') that runs estimate + budget checks together, plus feedback loops (stop, present numbers, tighten query/model/sampling, re-estimate on update) for this batch/autopay operation — satisfying the destructive/batch validation requirement.	5 / 5
Progressive Disclosure	Well-organized into clearly headed sections with no bulk reference material inlined that belongs in a separate file; navigation is easy. No bundle files exist and the procedural content is appropriately self-contained, but at ~175 lines it exceeds the under-50-line simple-skill exception that would allow a 5 without external references.	4 / 5
	Total	18 / 20 Passed

Description

95%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, specific description that clearly states what the skill does and when to trigger it, with helpful negative-trigger boundaries. The only weakness is mixed voice — third-person in the main clause but second-person ('you') in the trigger condition — which the rubric penalizes.

Suggestions

Rewrite the trigger clause in third person to match the opening ('Guides agents through...'), e.g. 'TRIGGER when: the user asks to create or configure a Replay Vision scanner, or the agent is about to call vision-scanners-create...' to avoid the second-person voice penalty.

Consider trimming the final sentence ('the skill's first section covers when that applies') since the inline-scan redirect is already covered by the DO NOT TRIGGER clause.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions ('choosing the scanner type (monitor/classifier/scorer/summarizer)', 'shaping the RecordingsQuery', 'estimating the credits', 'checking the org's remaining budget before creating') for comprehensive coverage, but the trigger clause uses second person ('when you are about to call vision-scanners-create'), so the rubric's voice penalty caps this at 4 rather than 5.	4 / 5
Completeness	Explicitly answers both 'what' (create and safely size a scanner, pick type, shape query, estimate credits, check budget) and 'when' via a concrete 'TRIGGER when' clause, plus negative trigger guidance.	5 / 5
Trigger Term Quality	Comprehensive natural-term coverage with synonyms ('create, set up, or configure') plus explicit tool names (vision-scanners-create, vision-scanners-update) and a DO NOT TRIGGER list that captures common mis-firings.	5 / 5
Distinctiveness Conflict Risk	A clearly defined niche (Replay Vision scanner creation/sizing with budget guardrails) with a DO NOT TRIGGER list that disambiguates reading, deleting, on-demand, and inline-scan use, minimizing conflict risk.	5 / 5
	Total	19 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: PostHog/posthog
Path: products/replay_vision/skills/creating-replay-vision-scanners/SKILL.md
Commit: 5b94e47

Reviewed: about 10 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.