Name: feature-usage-feed
Rating: 70.39999999999999 (1 reviews)
Author: PostHog

feature-usage-feed

Set up an LLM-judge evaluation that extracts canonical use cases for a PostHog feature at scale and streams the results to a Slack channel as a live feed. Use when someone wants to understand how users are actually using a specific AI/LLM-powered feature in production — what they're investigating, what questions they're trying to answer, and what patterns surface — without manually reading hundreds of traces. Assumes the feature emits `$ai_generation` and `$ai_evaluation` events with `$session_id` linkage to the trigger user's recording (the standard setup post the session-summary linkage PRs).

Quality

85%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Quality

Content

77%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A highly actionable, well-sequenced workflow with strong validation checkpoints for a batch/automation operation. Its main weakness is length and monolithic structure: worked examples and detailed payloads are inlined rather than split into reference files, and some narrative is redundant.

Suggestions

Move the two worked examples (A and B) into a separate references file and summarize them in one or two lines in the body, since they largely restate the filter patterns and PR history already in the workflow steps.

Extract the full Slack block-kit JSON and the eval-create/run/update payloads into a references/ file, keeping only the load-bearing fields and the UI-fallback path inline.

Trim the PR-history narrative in 'Two filter patterns' and the prerequisites to the minimum needed to verify linkage; the deployment/PR details add length without changing what Claude must do.

Dimension	Reasoning	Score
Conciseness	The body is dense and mostly actionable, but at ~445 lines it includes redundant narrative — worked examples A/B re-explain filter patterns and PR history already covered in the workflow, and some rationale could be trimmed.	3 / 5
Actionability	Fully executable guidance throughout: ready-to-run SQL queries, complete JSON payloads for each MCP tool, a full Slack block-kit template, and concrete UI navigation paths, with both Pattern A and B covered copy-paste ready.	5 / 5
Workflow Clarity	A clearly sequenced 7-step workflow with explicit validation checkpoints (create disabled, dry-run on samples, verify stored result distribution via SQL before saving, end-to-end production verify) and feedback loops (iterate prompt against a failure-mode table, synthetic-then-real test panel).	5 / 5
Progressive Disclosure	Well-organized into clear sections, but the entire ~445-line skill is a single monolithic file with no bundle references — worked examples, full JSON/SQL payloads, and the tips section are inlined content that could plausibly live in separate reference files.	3 / 5
	Total	16 / 20 Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, third-person description that concretely states what the skill does and gives explicit, natural-language trigger guidance. It is specific, well-scoped, and clearly distinct from neighboring skills.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions — 'Set up an LLM-judge evaluation', 'extracts canonical use cases', 'streams the results to a Slack channel as a live feed' — with comprehensive coverage of what the skill does, all in third person.	5 / 5
Completeness	Explicitly answers both 'what' (set up an LLM-judge eval that extracts use cases and streams them to Slack) and 'when' (a concrete 'Use when someone wants to understand...' clause with specific trigger phrasing).	5 / 5
Trigger Term Quality	Includes natural phrases users would say ('how users are actually using', 'what they're investigating', 'what questions they're trying to answer', 'what patterns surface'), though a few common synonyms or shorthand variants are absent.	4 / 5
Distinctiveness Conflict Risk	Occupies a clear niche — PostHog feature usage via LLM-judge eval piped to a Slack feed — with distinctive triggers that are unlikely to fire for unrelated skills.	5 / 5
	Total	19 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: PostHog/posthog
Path: products/ai_observability/skills/feature-usage-feed/SKILL.md
Commit: 6fca5f8

Reviewed: 3 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.