CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

signals-scout-ai-observability

Signals scout for PostHog AI observability. Watches LLM traces for cost, latency, error, volume, and eval-performance regressions.

58

Quality

73%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

—

The risk profile of this skill

Fix and improve this skill with Tessl

tessl review fix ./products/signals/skills/signals-scout-ai-observability/SKILL.md
SKILL.md
Quality
Evals
Security

Quality

Content

71%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is a highly actionable, well-sequenced scouting workflow with concrete tools, key patterns, and validation checkpoints. Its weaknesses are verbosity in the decision section, a duplicated tool entry, and a dangling `references/lenses.md` reference that undermines progressive disclosure.

Suggestions

Create the referenced `references/lenses.md` (or remove the dangling reference) so the per-lens playbook pointer resolves to a real file.

Split the long Decide paragraph into a short edit-vs-author decision table or checklist to reduce the wall-of-text and raise workflow clarity and conciseness.

De-duplicate the `scout-members-list` entry in the MCP tools section and tighten the repeated edit-vs-author guidance to recover token budget.

DimensionReasoningScore

Conciseness

Mostly operational specifics that earn their tokens, but noticeably verbose in places — the run-on Decide paragraph and a duplicated `scout-members-list` entry (lines 146 and 154) could be tightened, fitting anchor 3 rather than 4.

3 / 5

Actionability

Fully actionable: concrete MCP tool calls (`query-llm-trace`, `execute-sql`, `scout-emit-report`), exact scratchpad key patterns (`pattern:llm_analytics:generation-baseline`), and specific field values (`actionability=immediately_actionable`, `suggested_reviewers` as a list of objects) cover the common cases.

5 / 5

Workflow Clarity

Clear sequenced sections (Get oriented → Explore → Dig in → Save memory → Decide → Close out) with explicit validation checkpoints (check inbox, report status, scratchpad pointers), but the dense Decide paragraph and disqualifier interplay leave minor clarity gaps versus the checklist-level anchor 5.

4 / 5

Progressive Disclosure

References are clearly signaled ("references/lenses.md is the per-lens playbook — read it for...") but that file does not exist in the bundle, and substantial per-lens and decision detail is inlined rather than split out, matching anchor 3.

3 / 5

Total

15

/

20

Passed

Description

66%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific and well-targeted to a clear PostHog AI-observability niche, with good natural trigger terms. Its main weakness is the absence of an explicit "Use when..." clause, which caps completeness at 3.

Suggestions

Add an explicit trigger clause, e.g. "Use when running a PostHog Signals scout pass over a team's LLM/AI observability data."

Broaden trigger terms with synonyms users actually say (e.g. "LLM monitoring", "AI spend", "token cost", "trace latency").

Lead with one or two more concrete verbs beyond "Watches" (e.g. "Detects and reports ... regressions") to lift specificity toward 5.

DimensionReasoningScore

Specificity

Names the domain and several concrete surfaces it watches ("cost, latency, error, volume, and eval-performance regressions"), but the single action verb "Watches" keeps it just below the comprehensive multi-action anchor 5.

4 / 5

Completeness

It clearly states what the skill does ("Watches LLM traces for ...") but has no "Use when..." clause or equivalent explicit trigger guidance, so completeness is capped at 3 per the judging guidelines.

3 / 5

Trigger Term Quality

Natural domain terms a user would say are present ("LLM traces", "cost", "latency", "eval-performance", "PostHog AI observability"), but a few common synonyms and file/extension-style triggers are missing, fitting anchor 4.

4 / 5

Distinctiveness Conflict Risk

"Signals scout for PostHog AI observability" carves a clear niche with distinct triggers and only minor overlap risk with adjacent PostHog scouting skills, matching anchor 4 rather than the fully-distinct anchor 5.

4 / 5

Total

15

/

20

Passed

Validation

81%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 13 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

metadata_version

'metadata.version' is missing

Warning

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

referenced_paths_exist

Referenced path issues: 2 missing

Warning

Total

13

/

16

Passed

Repository

PostHog/posthog

Path

products/signals/skills/signals-scout-ai-observability/SKILL.md

Revision

130f3a1

Reviewed

Table of Contents

DiscoveryImplementationValidation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.

Claim skill