Name: feature-usage-feed
Rating: 77.60000000000001 (1 reviews)
Author: PostHog

feature-usage-feed

Set up an LLM-judge evaluation that extracts canonical use cases for a PostHog feature at scale and streams the results to a Slack channel as a live feed. Use when someone wants to understand how users are actually using a specific AI/LLM-powered feature in production — what they're investigating, what questions they're trying to answer, and what patterns surface — without manually reading hundreds of traces. Assumes the feature emits `$ai_generation` and `$ai_evaluation` events with `$session_id` linkage to the trigger user's recording (the standard setup post the session-summary linkage PRs).

Quality

96%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Quality

Content

92%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is highly actionable and operationally rich, with a clearly sequenced workflow and strong validation feedback loops — its only structural weakness is that it is a single large file with no progressive disclosure via references.

Suggestions

Move the two Worked Examples (group session summary; error-tracking mode) into a references/ file such as WORKED_EXAMPLES.md and link to them from the Workflow section, keeping SKILL.md as the overview.

Extract the Slack block-kit JSON template (Step 6.3) and the LLM-judge prompt template (Step 3) into separate one-level-deep reference files to reduce inline bulk.

Add a short top-of-file Quick-start summary that links to the detailed sections, giving the large monolithic skill clearer top-level navigation.

Dimension	Reasoning	Score
Conciseness	The body is dense with non-obvious operational gotchas Claude does not know (the stored 'True'/'False'/'None' vs 'PASS'/'FAIL' distinction, synthetic-vs-real Workflows test modes, the 3-6MB context-window warning) and never condescends to explain basics; a little redundancy (topic drift appears in both Step 2 and Worked Example B) is not enough to drop it below the lean/efficient anchor.	3 / 3
Actionability	Provides fully executable SQL queries, complete MCP tool JSON payloads, and copy-paste Slack block-kit JSON, all correctly parameterized with <placeholders> rather than pseudocode, matching the copy-paste-ready anchor.	3 / 3
Workflow Clarity	Seven steps are clearly sequenced with explicit validation checkpoints and feedback loops — coverage/volume checks in Step 1, iterate-on-3-5-traces in Step 4, synthetic→real testing in §6.4, and a 4-point end-to-end production checklist with error-recovery guidance in Step 7.	3 / 3
Progressive Disclosure	No bundle files exist and the ~430-line skill is monolithic: the two worked examples, the Slack block-kit template, and the LLM-judge prompt template are inline material that could be split into one-level-deep reference files, fitting the 'content that should be separate is inline' anchor rather than the well-split score-3 anchor.	2 / 3
	Total	11 / 12 Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific, third-person, and cleanly answers both what the skill does and when to use it with natural trigger phrasing. It occupies a distinctive niche with low conflict risk against other skills.

Dimension	Reasoning	Score
Specificity	Quotes "Set up an LLM-judge evaluation that extracts canonical use cases ... and streams the results to a Slack channel as a live feed" — names multiple concrete actions (set up the eval, extract canonical use cases, stream to Slack) rather than vague language, matching the score-3 anchor.	3 / 3
Completeness	Both halves are explicit: the "Set up ..." clause answers what the skill does and the "Use when ..." clause answers when Claude should invoke it, satisfying the what-AND-when anchor with explicit triggers.	3 / 3
Trigger Term Quality	Quotes "Use when someone wants to understand how users are actually using a specific AI/LLM-powered feature in production ... without manually reading hundreds of traces" — natural phrases a user would actually say, with good coverage of variants (use cases, Slack feed, traces).	3 / 3
Distinctiveness Conflict Risk	A clear niche — building a PostHog LLM-judge usage feed streamed to Slack — with distinctive triggers unlikely to fire for unrelated skills; the body even redirects near-miss intents to sibling skills, reinforcing non-conflict.	3 / 3
	Total	12 / 12 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: PostHog/posthog
Path: products/ai_observability/skills/feature-usage-feed/SKILL.md
Commit: a5d244e

Reviewed: about 13 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.