CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

flake-dashboard-author

Use when a team needs a lasting view of test reliability from CI history (JUnit XML or JSON): a flake-rate metric, a Grafana panel or Datadog CI Visibility view, the quarantine-candidate query, trend alerts, and a weekly or monthly test-health report for Slack or a wiki.

Quality

79%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./plugins/qa-flake-triage/skills/flake-dashboard-author/SKILL.md

The canonical home for this skill is testland/flake-dashboard-author

SKILL.md
Quality
Evals
Security

Quality

Content

57%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is rich in concrete, copy-paste artifacts and clearly sequenced, but it is held back by a monolithic single-file layout with broken references to nonexistent bundle scripts, missing validation loops around batch ingestion, and redundant sections (terminology note, duplicated references) that pad the token budget.

Suggestions

Remove the opening terminology note (ISTQB/Google-blog definitions of 'flaky test', 'failure', 'test run' — concepts Claude already knows) and the closing References section that restates links already defined inline, to cut padding (conciseness).

Add validation checkpoints and error-recovery loops for the batch/database steps: after ingestion, verify expected row counts and status values before computing flake rates, and give a fix path if the Grafana panel renders empty (e.g., check the ${DS_POSTGRES} datasource UID) (workflow_clarity).

Ship the referenced bundle files (scripts/parse_junit.py, scripts/ingest_playwright_json.js, scripts/quarantine_candidates.sql, dashboards/flakiness-overview.json) or remove those invocations, and move the full panel JSON and report template into a references/ file so SKILL.md stays an overview (progressive_disclosure, actionability).

DimensionReasoningScore

Conciseness

The body is dense and mostly actionable, but the opening terminology note ("'flaky test' is a practitioner-emergent term... ISTQB Glossary v4.7.1 definitions") explains concepts Claude already knows, and the closing References section restates link definitions already given inline. Fits the score-3 anchor ("mostly efficient but includes some unnecessary explanation or could be tightened"); it is not score 4 because the redundant material spans two full sections rather than minor instances.

3 / 5

Actionability

Mostly executable guidance: complete SQL DDL, a copy-paste Grafana panel JSON, the quarantine-candidate query, and numbered alert-rule steps. It falls short of the score-5 fully-executable anchor because commands like "python3 scripts/parse_junit.py", "node scripts/ingest_playwright_json.js", and "-f scripts/quarantine_candidates.sql" invoke script files that do not exist in the bundle (no scripts/ directory is present), so those steps fail as written.

4 / 5

Workflow Clarity

Steps 1-6 are clearly sequenced ("Define the flake-rate metric" → "Build the data model" → "Author the Grafana panel JSON" → ...) with scattered checkpoints ("Verify metric existence in Metrics Explorer before building the Timeboard widget", the "HAVING COUNT(*) >= 10" guard). However, the batch ingestion and database operations have no validate→fix→retry feedback loop (no row-count or status-value verification after loading runs), which caps this dimension at 3 per the rubric's batch/database guidance.

3 / 5

Progressive Disclosure

Section structure and headers are good, but everything is inlined in a single ~410-line file — the full panel JSON, the quarantine SQL, and a ~35-line report template — where the rubric expects bulk material in one-level-deep reference files. Additionally, referenced paths ("scripts/parse_junit.py", "scripts/ingest_playwright_json.js", "dashboards/flakiness-overview.json") point to files absent from the bundle, so navigation to supporting material is broken; this fits score 3 ("content that should be separate is inline") rather than 4.

3 / 5

Total

13

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it explicitly states both the trigger scenario and a comprehensive list of concrete deliverables with named tools and formats. The only gap is missing natural synonyms like "flaky tests" or "flakiness" that a user might say verbatim.

DimensionReasoningScore

Specificity

The description enumerates five concrete deliverables — "a flake-rate metric, a Grafana panel or Datadog CI Visibility view, the quarantine-candidate query, trend alerts, and a weekly or monthly test-health report" — with named tools (Grafana, Datadog) and input formats ("JUnit XML or JSON"). This matches the comprehensive-coverage anchor; it goes beyond the score-4 anchor ("several specific actions; minor gaps") because the deliverable list covers the skill's full scope.

5 / 5

Completeness

Both halves are explicit: the when is "Use when a team needs a lasting view of test reliability from CI history (JUnit XML or JSON)" and the what is the colon-delimited list of concrete outputs. This matches the score-5 anchor (clearly and explicitly answers both what AND when with concrete trigger phrases) rather than score 4, where the 'when' would be present but only vaguely specified.

5 / 5

Trigger Term Quality

Natural terms are well covered: "test reliability", "CI history", "JUnit XML", "flake-rate", "Grafana", "Datadog CI Visibility", "quarantine", "trend alerts". However, common synonyms users would actually say — "flaky tests", "flakiness", "test stability" — are absent from the description, so it falls short of the score-5 anchor requiring comprehensive synonyms; it is clearly above score 3 ("some relevant keywords but missing common variations") given the breadth of tool and artifact terms.

4 / 5

Distinctiveness Conflict Risk

The CI test-reliability observability niche — flake-rate metrics from JUnit history rendered in Grafana/Datadog with quarantine queries — is a clear niche with distinct triggers, minimal overlap with generic dashboard, reporting, or document skills. Matches the score-5 anchor; nothing here would plausibly fire the wrong skill.

5 / 5

Total

19

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

referenced_paths_exist

Referenced path issues: 3 missing

Warning

Total

15

/

16

Passed

Repository

testland/qa

Path

plugins/qa-flake-triage/skills/flake-dashboard-author/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.