CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

agent-host-e2e-tests

Use when writing, recording, updating, validating, or troubleshooting the agent host end-to-end tests under src/vs/platform/agentHost/test/node/e2e (black-box tests that drive the whole agent host over the AHP protocol, using a CapiReplayProxy record/replay system for Claude/Copilot/Codex). Covers adding a cross-provider test, re-recording fixtures after an SDK bump, cross-platform Azure validation, gating non-deterministic or platform-specific tests, and diagnosing replay cache misses.

80

Quality

100%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

The canonical home for this skill is agent-host-e2e-tests in microsoft/vscode

SKILL.md
Quality
Evals
Security

Quality

Content

100%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is a lean, highly actionable operations manual: non-negotiable rules, four sequenced workflows with explicit validation gates, and a troubleshooting section that defers detail to the README. Structure and delegation are exemplary, and nothing is explained that Claude could infer on its own.

DimensionReasoningScore

Conciseness

The body contains zero conceptual padding — no explanations of things Claude already knows — and every line is project-specific policy, a concrete command, or a pointed delegation ('This skill is only the *workflows*; the README is the source of truth for *how it works*'), matching 'every token earns its place' rather than the 4-anchor's trimmable over-explanation.

5 / 5

Actionability

Fully executable throughout: copy-paste commands ('AGENT_HOST_REPLAY_RECORD=1 ./scripts/test-integration.sh --run src/…', 'gh auth token', '--grep "<test title fragment>"', pipeline '111' with 'VSCODE_BUILD_TYPE=CI') and exact API/gate forms ('hostOnlyTest(context, …)', 'dispatchTurn(…)', "(RECORD ? test : test.skip)('…')", '!isWindows') cover the common cases.

5 / 5

Workflow Clarity

Four workflows are clearly sequenced with explicit validation checkpoints and feedback loops — 'Review the diff… then run the test in plain replay mode to confirm it's green', 'Treat the Agent Host E2E result as accepted only when the Electron integration tests succeed on Windows, Linux, and macOS', 'require two clean executions of every new test on each supported platform' — and the real-CAPI recording and batch pipeline runs are validated, so no cap applies.

5 / 5

Progressive Disclosure

The body keeps only the workflows and delegates architecture, fixture format, config flags, and troubleshooting to the e2e README via a top-of-file, clearly signaled one-level reference ('Before doing anything, read the architecture + troubleshooting reference'), plus focused pointers to KNOWN_ISSUES.md, capiReplayProxy.ts, and capiStubs.ts — well-sectioned and easy to navigate.

5 / 5

Total

20

/

20

Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is a model example: it states what the skill covers in concrete, verifiable terms and gives an explicit 'Use when…' clause with natural trigger phrases tied to the exact directory. Distinctiveness is essentially perfect because it names the specific codebase path and subsystem.

DimensionReasoningScore

Specificity

Lists multiple concrete capabilities both as verbs ('writing, recording, updating, validating, or troubleshooting') and as covered tasks ('adding a cross-provider test, re-recording fixtures after an SDK bump, cross-platform Azure validation, gating non-deterministic or platform-specific tests, and diagnosing replay cache misses'), with comprehensive coverage of everything the body addresses — matches the top anchor rather than the 4-anchor's 'minor gaps in coverage'.

5 / 5

Completeness

Explicitly answers 'when' ('Use when writing, recording, updating, validating, or troubleshooting…') with concrete trigger phrases and 'what' ('black-box tests that drive the whole agent host over the AHP protocol, using a CapiReplayProxy record/replay system') — both halves explicit, so it sits at the 5-anchor rather than the 4-anchor's 'when could be more explicit'.

5 / 5

Trigger Term Quality

Natural terms a developer would actually say are comprehensively covered — 'end-to-end tests', 'e2e' (in the path), 're-record fixtures', 'SDK bump', 'replay cache misses', 'cross-platform Azure validation' — alongside the concrete directory path and the Claude/Copilot/Codex provider names, hitting the synonym-level coverage of the 5-anchor.

5 / 5

Distinctiveness Conflict Risk

Names a specific test directory and system (AHP protocol, CapiReplayProxy) that no other skill would trigger on — a clear niche with distinct triggers and minimal conflict risk, matching the top anchor.

5 / 5

Total

20

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

posit-dev/positron

Path

.github/skills/agent-host-e2e-tests/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.