CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

testland/flake-dashboard-author

Use when a team needs a lasting view of test reliability from CI history (JUnit XML or JSON): a flake-rate metric, a Grafana panel or Datadog CI Visibility view, the quarantine-candidate query, trend alerts, and a weekly or monthly test-health report for Slack or a wiki.

Quality

79%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

57%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body delivers genuinely specific, tool-accurate guidance (executable SQL, a complete Grafana panel JSON, real Datadog queries and monitor settings) organized as a clear six-step sequence. Its weaknesses are dangling references to scripts and dashboard files that are not in the bundle, missing validation checkpoints around the batch ingestion and dashboard-import steps, and padding such as the terminology note and the fully worked mock report.

Suggestions

Ship the referenced bundle files (scripts/parse_junit.py, scripts/ingest_playwright_json.js, scripts/quarantine_candidates.sql, dashboards/flakiness-overview.json) or remove their invocations from the worked example — every referenced path currently resolves to nothing.

Add validation checkpoints after the batch operations: verify row counts in test_runs after ingestion (e.g. a SELECT COUNT(*) sanity check against the expected number of test cases) and confirm the imported Grafana dashboard renders before wiring alerts.

Trim the lowest-value prose: the ISTQB/Google-blog terminology note, the invented Notes/Suggested-follow-ups inside the mock report, and the link-by-link annotations in References — the URLs and Steps already carry that context.

DimensionReasoningScore

Conciseness

The core material (formula, DDL, panel JSON, queries, monitor steps) is dense and task-specific, but several sections pad it: the ISTQB/Google-blog "Terminology note" explaining what a flaky test is, a full mock trend report with invented numbers and follow-ups, a trend-arrow legend, and a References section re-explaining each link. This matches "mostly efficient but includes some unnecessary explanation or could be tightened" rather than the minor-trimming 4 anchor.

3 / 5

Actionability

The SQL schema, Grafana panel JSON, Datadog queries, and monitor configuration are concrete and executable, but the worked example invokes scripts/ingest_playwright_json.js, scripts/quarantine_candidates.sql, and dashboards/flakiness-overview.json that are absent from the bundle, and Playwright JSON ingestion is described ("iterating results[].suites[].specs[].tests[].results[]") rather than coded. Mostly executable with minor gaps — the 4 anchor, not the fully copy-paste-ready 5.

4 / 5

Workflow Clarity

Steps 1-6 are clearly sequenced (metric → data model → dashboard → alternative → quarantine query → alerting), and there are partial checkpoints (the HAVING COUNT(*) >= 10 sample guard, Datadog's "Verify metric existence in Metrics Explorer"). However the database batch-ingestion pipeline and dashboard import have no validation or feedback loop — no row-count check after ingest, no verify step after import — and the rubric's scoring note caps database-operation workflows without feedback loops at 3.

3 / 5

Progressive Disclosure

Section structure and step navigation are good, but the bundle ships zero files while the body references four non-existent paths (scripts/parse_junit.py, scripts/ingest_playwright_json.js, scripts/quarantine_candidates.sql, dashboards/flakiness-overview.json), and bulk artifacts (the 40-line panel JSON, the full report template, the long quarantine SQL) are inlined rather than split into reference files. This fits "some structure but could be better organized; content that should be separate is inline" — better than the headerless monolithic 2 anchor because headers and sequencing are clear.

3 / 5

Total

13

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description that explicitly pairs a concrete when-clause with a comprehensive list of specific deliverables and named tools. Its only weakness is trigger-term coverage: the plain user phrasings "flaky tests" and "flakiness" are only reachable via the compound "flake-rate metric".

DimensionReasoningScore

Specificity

Enumerates five concrete deliverables — "a flake-rate metric, a Grafana panel or Datadog CI Visibility view, the quarantine-candidate query, trend alerts, and a weekly or monthly test-health report" — comprehensively covering the skill's actual scope with no gaps, matching the 5 anchor rather than 4.

5 / 5

Completeness

Explicitly answers both questions: "Use when a team needs a lasting view of test reliability from CI history (JUnit XML or JSON)" gives a concrete when-clause with format triggers, and the colon-delimited list fully states what the skill produces.

5 / 5

Trigger Term Quality

Good natural keyword coverage ("test reliability", "CI history", "JUnit XML or JSON", "Grafana", "Datadog", "quarantine", "test-health report"), but the most common user phrasing "flaky tests"/"flakiness" appears only indirectly as "flake-rate" and synonyms like "test stability" are absent, so it falls short of the comprehensive-synonyms 5 anchor.

4 / 5

Distinctiveness Conflict Risk

Clear niche (flake-rate observability from CI history) with named tools (Grafana, Datadog CI Visibility, JUnit XML) and quarantine-specific triggers; minimal overlap risk with generic test or reporting skills.

5 / 5

Total

19

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

referenced_paths_exist

Referenced path issues: 3 missing

Warning

Total

15

/

16

Passed

Reviewed

On this page