creating-replay-vision-scanners

Guides agents through creating and safely sizing a Replay Vision scanner: choosing the scanner type (monitor/classifier/scorer/summarizer), shaping the RecordingsQuery that selects sessions, and — crucially — estimating the credits it will spend and checking the org's remaining budget before creating, so a broad scanner doesn't exhaust the budget on its first scheduled sweep. TRIGGER when: user asks to create, set up, or configure a Replay Vision scanner, OR when you are about to call vision-scanners-create, OR when widening an existing scanner's query, sampling_rate, or sampling_mode (or moving it to a pricier model) via vision-scanners-update. DO NOT TRIGGER when: only reading scanners or observations, deleting a scanner, or running an existing scanner against a single session on demand (vision-scanners-scan-session). For a one-off question about sessions you already have, use vision-scanners-inline-scan-create rather than creating a scanner — the skill's first section covers when that applies.

Quality

88%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

—

The risk profile of this skill

Quality

Content

81%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, actionable workflow skill with a strong mandatory budget-validation checkpoint and re-estimate feedback loop. It is mostly lean and executable, with only minor conciseness trims and a missing code example for the estimate/quota calls.

Suggestions

Show a concrete example call for vision-scanners-estimate-create and vision-quota-retrieve (arguments and a sample response) alongside the proration formula, mirroring the copy-paste create example.

Tighten the framing prose (e.g. "That schedule is exactly why creation needs a gut-check") so the body leans more heavily on directives and tables.

Consider extracting the per-type scanner_config reference into a short reference table or file so the main flow reads as a flow rather than a schema catalog.

Dimension	Reasoning	Score
Conciseness	Largely lean and assumes Claude's competence; explanations are domain-specific (credits, the 5-minute sweep, budget mechanics) rather than generic padding, though a few prose passages (e.g. the "That schedule is exactly why..." framing) could be trimmed slightly.	4 / 5
Actionability	Provides a copy-paste-ready JSON create example, concrete scanner_config shapes per type, named estimate/quota return fields, and a concrete proration formula — mostly executable, but the estimate and quota calls are described rather than shown as code, leaving a minor gap.	4 / 5
Workflow Clarity	Steps 1-4 are clearly sequenced with an explicit, must-not-skip validation checkpoint (Step 3 gut-check comparing projected spend to remaining budget) and a feedback loop (re-run the gut-check whenever widening scope or raising price on update), satisfying the top anchor for a budget-affecting operation.	5 / 5
Progressive Disclosure	No bundle files exist; the skill is a single cohesive file with well-organized headers and one-level external references to sibling skills, which is good structure but not the idealized multi-file quick-start/referenced-detail split the top anchor describes.	4 / 5
	Total	17 / 20 Passed

Description

95%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, specific description with explicit what/when triggers and clear boundary guidance against adjacent operations. The only weakness is second-person phrasing in the trigger clause, which costs one specificity point per the voice guideline.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions ("choosing the scanner type (monitor/classifier/scorer/summarizer)", "shaping the RecordingsQuery", "estimating the credits", "checking the org's remaining budget"), approaching comprehensive coverage; reduced from 5 because the TRIGGER clause uses second person ("when you are about to call"), which the voice guideline penalizes by one point.	4 / 5
Completeness	Explicitly answers both what ("creating and safely sizing a Replay Vision scanner") and when (a concrete "TRIGGER when:" clause with multiple trigger phrases), and adds a "DO NOT TRIGGER when:" boundary — clearly matching the top anchor.	5 / 5
Trigger Term Quality	Covers natural synonyms ("create, set up, or configure a Replay Vision scanner") plus specific tool names (vision-scanners-create, vision-scanners-update, vision-scanners-inline-scan-create), giving comprehensive coverage of natural terms users would say.	5 / 5
Distinctiveness Conflict Risk	The niche is sharply defined by the Replay Vision domain and named tools, and the explicit "DO NOT TRIGGER" disambiguation against inline-scan/read/delete keeps conflict risk minimal.	5 / 5
	Total	19 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: PostHog/posthog
Path: products/replay_vision/skills/creating-replay-vision-scanners/SKILL.md
Commit: 130f3a1

Reviewed: 9 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.