CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

agent-host-e2e-tests

Use when writing, recording, updating, validating, or troubleshooting Agent Host end-to-end and real-provider integration tests under src/vs/platform/agentHost/test/node/e2e and providerIntegration. Covers adding a cross-provider test, re-recording fixtures after an SDK bump, required cross-platform Azure validation, gating non-deterministic or platform-specific tests, and diagnosing replay cache misses.

Quality

91%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

81%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

This is a strong, curated operational skill: workflows are precisely sequenced with explicit validation and feedback loops, commands are concrete and executable, and detail is deliberately deferred to a README and sibling skills. The main improvement opportunities are trimming the inline Copilot log mechanics and long best-practice bullets, and inlining the canonical Azure pipeline command.

Suggestions

Move the Copilot runtime-log mechanics (the long troubleshooting bullet) into the E2E README's troubleshooting section and keep only the pointer — the skill's own curation rule says failure-specific logs and mechanics belong there.

Inline the canonical Azure pipeline definition-111 queue command (or a one-line copy-paste example) in the cross-platform validation section instead of deferring entirely to the azure-pipelines skill, so the workflow is self-contained when that skill isn't loaded.

Tighten the longest best-practice bullets (scope/lifecycle and storage semantics) to their actionable core, moving rationale into the README.

DimensionReasoningScore

Conciseness

The body is dense and assumes Claude's competence (no concept explanations, every workflow step operational), but the ~120-word Copilot runtime-log mechanics on the troubleshooting line duplicate a README section the skill itself cites, and a few best-practice bullets (storage semantics, scope/lifecycle) run long — minor instances of over-explanation that could be trimmed rather than 'every token earns its place'.

4 / 5

Actionability

Most guidance is copy-paste executable: the record command with AGENT_HOST_REPLAY_RECORD=1 and a concrete test path, gh auth token, --grep filtering, the (RECORD ? test : test.skip) gate pattern, !isWindows gates, and pipeline 111 with VSCODE_BUILD_TYPE=CI and test ID integration-agent-host-e2e. The one notable gap is deferring the canonical Azure queue command entirely to the azure-pipelines skill.

4 / 5

Workflow Clarity

Four workflows (write/evaluate, record/re-record, gating, Azure validation) are explicitly sequenced with validation checkpoints and feedback loops: review git diff on fixtures, run plain replay to confirm green, extend redaction and re-record on leaks, two clean executions per platform, and rerunning unrelated failures in isolation before attributing them.

5 / 5

Progressive Disclosure

The split is well declared ('This skill is only the workflows; the README is the source of truth') with one-level-deep, clearly signaled references (E2E README, KNOWN_ISSUES.md, ci-artifact-testing, azure-pipelines skills). The minor gap is that the inline Copilot runtime-log mechanics belong in the README per the skill's own curation rule, keeping this just below the top anchor.

4 / 5

Total

17

/

20

Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is a model example: it pairs an explicit 'Use when…' trigger clause with a comprehensive, concrete list of covered tasks, all in third-person imperative form with no fluff or over-claims. Trigger terms map directly onto what a developer in this repository would naturally say.

DimensionReasoningScore

Specificity

The description lists multiple concrete, distinct actions — "re-recording fixtures after an SDK bump", "required cross-platform Azure validation", "gating non-deterministic or platform-specific tests", and "diagnosing replay cache misses" — giving comprehensive coverage of the skill's capabilities, matching the top anchor rather than the 'minor gaps' level below.

5 / 5

Completeness

It explicitly answers both questions: an explicit "Use when writing, recording, updating, validating, or troubleshooting…" trigger clause plus a "Covers…" list detailing what the skill does, with concrete trigger phrases throughout.

5 / 5

Trigger Term Quality

It covers the natural terms a user in this domain would say — "end-to-end", "integration tests", "re-record fixtures", "SDK bump", "cache miss", "gating", "cross-platform" — plus a concrete directory path, leaving no meaningful natural term or synonym absent.

5 / 5

Distinctiveness Conflict Risk

It is anchored to a specific niche (Agent Host E2E/real-provider integration tests under an exact source path) with distinct triggers like replay cache misses and Azure validation, making conflict with sibling skills minimal.

5 / 5

Total

20

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

relative_links

Relative link issues: 1 suspicious

Warning

Total

15

/

16

Passed

Repository

microsoft/vscode

Path

.github/skills/agent-host-e2e-tests/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.