CtrlK
BlogDocsLog inGet started
Tessl Logo

flaky-smoke-tests

Diagnose intermittent VS Code Electron smoke-test failures from the Azure DevOps Flaky Smoke Tests pipeline (definition 700). Covers finding failed iterations, downloading task logs and platform artifacts with Azure CLI, correlating cumulative runner logs, tracing the introducing commit, and queueing focused validation runs.

70

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Low

Low-risk findings worth noting

SKILL.md
Quality
Evals
Security

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A thorough, executable runbook with a clear sequenced workflow and strong validation checkpoints. The only gaps are minor verbosity in the correlation/UI sections and the absence of any split-out reference files for lengthy reference material.

Suggestions

Consider moving the platform artifact table and the detailed UI-diagnostic pitfall list into a reference file to keep the core workflow tighter.

Tighten the bullet explanations in sections 5 and 6 to their actionable essence to recover token budget.

DimensionReasoningScore

Conciseness

The body is dense with non-obvious domain gotchas (artifact scoping, log overwrite risks, UI race pitfalls) that earn their tokens and largely assumes Claude's competence; a few explanatory bullets in sections 5–6 could be trimmed.

4 / 5

Actionability

Copy-paste ready az CLI, git, jq, and PowerShell commands with clearly marked placeholders cover the common diagnostic cases end-to-end.

5 / 5

Workflow Clarity

Nine clearly sequenced sections with explicit validation checkpoints (confirm sourceVersion, §9 local validation, re-check the gesture/dispatch/request chain) and feedback loops for the batch/run-cancel operations.

5 / 5

Progressive Disclosure

No bundle files exist; content is organized into well-labeled sections within a single cohesive workflow file with no nested references, though some reference-grade detail (platform table, UI pitfalls) is inlined rather than split out.

4 / 5

Total

18

/

20

Passed

Description

83%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A highly specific, action-oriented description targeting a narrow pipeline-debugging niche with strong distinctiveness. Its main weakness is the absence of an explicit 'Use when…' trigger clause and a few missing natural synonyms.

Suggestions

Add an explicit 'Use when…' clause naming the trigger scenario, e.g. 'Use when diagnosing intermittent or flaky VS Code Electron smoke-test failures from the Azure DevOps Flaky Smoke Tests pipeline.'

Include a few more natural synonyms users might say ('flaky tests', 'CI failures', 'smoke suite') to broaden trigger coverage.

DimensionReasoningScore

Specificity

Lists five concrete actions — 'finding failed iterations, downloading task logs and platform artifacts with Azure CLI, correlating cumulative runner logs, tracing the introducing commit, and queueing focused validation runs' — giving comprehensive coverage of the diagnostic workflow.

5 / 5

Completeness

The 'what' is explicit and the 'when' is concretely embedded (intermittent failures from the definition-700 pipeline), but there is no explicit 'Use when…' clause, so it stops short of the top anchor.

4 / 5

Trigger Term Quality

Natural phrases a user would say are present ('intermittent VS Code Electron smoke-test failures', 'Flaky Smoke Tests pipeline', 'Azure DevOps'), but common synonyms like 'flaky tests' or 'CI' are absent.

4 / 5

Distinctiveness Conflict Risk

The niche is sharply scoped to VS Code Electron smoke-test failures on a specific Azure DevOps pipeline (definition 700), making conflict with other skills minimal.

5 / 5

Total

18

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository
posit-dev/positron
Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.