Name: exploring-llm-costs
Rating: 71.2 (1 reviews)
Author: PostHog

exploring-llm-costs

Investigate LLM spend in PostHog — total cost over time, cost by model, provider, user, trace, or custom dimension, token and cache-hit economics, and cost regressions. Use when the user asks "how much are we spending on LLMs?", "which model / user / feature is most expensive?", "why did cost spike?", wants to build a cost dashboard or alert, or pastes a trace URL and asks about its cost.

Quality

87%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

—

The risk profile of this skill

Quality

Content

75%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is well-organized, information-dense, and gives executable guidance for the primary workflows, with strong progressive-disclosure design. The main weaknesses are restated rules in the Tips section and that all referenced detail files are missing from the bundle, breaking the offloaded navigation.

Suggestions

Ship the referenced bundle files (cost-properties.md, cost-sources.md, cache-accounting.md, breakdown-patterns.md, regression-debugging.md, materializing.md) or inline their essential content so the links resolve.

De-duplicate the Tips section against Core rules — remove the 'Always set a time range' and 'Always include $ai_embedding' restatements and the repeated cache-flag guidance, keeping only the net-new tips.

Inline at least one representative breakdown recipe and the regression 5-step playbook so the most common workflows are actionable without chasing a missing reference.

Dimension	Reasoning	Score
Conciseness	The body is dense and assumes Claude's competence without explaining basics, but several Tips bullets restate Core rules (e.g. 'Always set a time range' and 'Always include $ai_embedding' appear twice, and the cache-flag branching is repeated), which could be trimmed.	4 / 5
Actionability	The total-spend SQL and the query-llm-trace JSON are copy-paste ready and the generate-app-url calls include concrete params, but the breakdown recipes, regression playbook, and materializing JSON live in referenced files that are not present, leaving gaps for those workflows.	4 / 5
Workflow Clarity	Workflows are laid out as clear sequenced sections (total spend, breakdowns, single trace, regression, materialize) with a 'surface a UI link to verify visually' checkpoint, though feedback loops are mostly implicit and the regression 5-step playbook is only referenced.	4 / 5
Progressive Disclosure	Good structure with a concise overview, well-signaled one-level-deep references, and a dedicated References list; however the six referenced ./references/*.md files (and the cross-skill reference) do not exist in the bundle, so the promised navigation does not resolve.	4 / 5
	Total	16 / 20 Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is exemplary: it concisely states concrete capabilities in third person and pairs them with explicit, natural-language trigger phrases covering the common cost questions. It hits the top anchor on every dimension with no fluff or over-claims.

Dimension	Reasoning	Score
Specificity	Names the domain (LLM spend in PostHog) and lists multiple concrete actions — total cost over time, breakdowns by model/provider/user/trace/custom dimension, token and cache-hit economics, cost regressions, dashboards/alerts, and per-trace cost — giving comprehensive coverage rather than vague language.	5 / 5
Completeness	Explicitly answers both halves: a clear 'what' (investigate LLM spend across the listed dimensions) and an explicit 'Use when...' clause with concrete trigger phrases, matching the top anchor exactly.	5 / 5
Trigger Term Quality	Quotes natural user phrases verbatim — "how much are we spending on LLMs?", "which model / user / feature is most expensive?", "why did cost spike?", plus "build a cost dashboard or alert" and pasting a trace URL — covering the common ways users actually ask for this.	5 / 5
Distinctiveness Conflict Risk	Occupies a clear niche (LLM cost analysis in PostHog) with distinctive triggers unlikely to fire for adjacent skills like trace inspection or expensive-user analysis.	5 / 5
	Total	20 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
relative_links	Relative link issues: 12 missing, 1 suspicious	Warning

	Total	15 / 16 Passed

Repository: PostHog/posthog
Path: products/ai_observability/skills/exploring-llm-costs/SKILL.md
Commit: 130f3a1

Reviewed: 9 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.