investigating-ci-failures

Investigates a specific CI failure to a verdict: whose fault, which commit, who wrote it, and whether it's fixed. Use for "who broke master", "why did this test fail in CI", "is this failure my PR's fault or everyone's", "is this test flaky or actually broken", "when did this failure start". Works from the engineering_analytics warehouse views (engineering_analytics_ci_failures, engineering_analytics_ci_job_history) plus the CI failure logs. Not for aggregate CI health, cost, or merge bottlenecks (use diagnosing-ci-and-merge-bottlenecks) and not for building saved insights (use turning-engineering-analytics-into-insights).

Quality

100%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Quality

Content

100%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A lean, information-dense skill body that supplies exactly the non-obvious operational knowledge Claude lacks (warehouse schemas, scan-pruning floors, merge-queue gate semantics, attribution rules) with executable SQL correctly split into a verified reference. Workflow is explicitly sequenced with validation checkpoints and feedback loops; structure is textbook progressive disclosure.

Dimension	Reasoning	Score
Conciseness	Dense and assumes Claude's competence throughout — it never explains what CI, pytest, or fingerprints are, and every prose block (cross-branch=trunk reasoning, the created_at_raw pruning trick, merge-queue gate semantics) is non-obvious domain logic that earns its tokens.	5 / 5
Actionability	Names specific MCP tools, exact view columns, concrete query references, and an ASCII boundary diagram, with copy-paste-ready executable SQL correctly delegated to the verified references/investigation-queries.md file — fully executable end to end.	5 / 5
Workflow Clarity	Clear sequence (fingerprint query 1 → shape table → trunk-boundary query 2 → culprit/fix, or flaky corroboration) with explicit validation checkpoints (confidence check before naming, window-mismatch detection, freshness check query 5, retry-attribution reading) and feedback loops (widen window + check adjacent commit).	5 / 5
Progressive Disclosure	SKILL.md is a well-structured overview (workflow, decision tables, caveats, surface-choice table) that delegates the bulk SQL to one clearly signaled, one-level-deep reference file which exists on disk and is itself well-organized — easy to navigate.	5 / 5
	Total	20 / 20 Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

An exemplary description: concrete outputs, natural verbatim trigger phrases, explicit what/when coverage, and active boundary-setting against neighboring skills. Every dimension sits at the top anchor with no drift toward neighboring levels.

Dimension	Reasoning	Score
Specificity	Names four concrete verdict outputs — "whose fault, which commit, who wrote it, and whether it's fixed" — plus the exact warehouse views it operates on, giving comprehensive concrete-action coverage rather than vague abstraction.	5 / 5
Completeness	Explicitly answers both what (investigates a CI failure to a verdict with named outputs and data sources) and when ("Use for ..." with concrete trigger phrases), matching the top anchor.	5 / 5
Trigger Term Quality	Five verbatim user-phrased triggers ("who broke master", "why did this test fail in CI", "is this failure my PR's fault or everyone's", "is this test flaky or actually broken", "when did this failure start") are exactly what a developer would say, with synonym coverage (flaky vs broken, my PR vs everyone's).	5 / 5
Distinctiveness Conflict Risk	Carves a clear niche and actively disambiguates from siblings ("Not for aggregate CI health, cost, or merge bottlenecks (use diagnosing-ci-and-merge-bottlenecks) and not for building saved insights"), minimizing wrong-skill triggers.	5 / 5
	Total	20 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: PostHog/posthog
Path: products/engineering_analytics/skills/investigating-ci-failures/SKILL.md
Commit: 6fca5f8

Reviewed: 3 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.