CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

agent-host-e2e-tests

Use when writing, recording, updating, validating, or troubleshooting Agent Host end-to-end and real-provider integration tests under src/vs/platform/agentHost/test/node/e2e and providerIntegration. Covers adding a cross-provider test, re-recording fixtures after an SDK bump, required cross-platform Azure validation, gating non-deterministic or platform-specific tests, and diagnosing replay cache misses.

76

Quality

96%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

92%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

An exceptionally dense, expert-level skill: every section encodes non-obvious operational rules, concrete commands, and explicit validation/feedback loops rather than generalities. The only notable gap is that the cross-platform Azure validation workflow, while fully parameterized, defers its queue command to the azure-pipelines skill.

Suggestions

Inline the canonical Azure pipeline-111 queue command (or a minimal working invocation with the listed VSCODE_BUILD_TYPE=CI and platform flags) in the 'Cross-platform Azure validation' section so that workflow is executable without loading the azure-pipelines skill.

DimensionReasoningScore

Conciseness

The body assumes Claude's competence throughout — no basic-concept explanations, only non-obvious operational knowledge like 'Fixtures are generated, never hand-edited', 'Separate storage semantics from device latency', and 'Keep temporary workspaces outside instruction-bearing ancestors'. Every token carries repo-specific guidance a general model could not reconstruct.

5 / 5

Actionability

Mostly executable: copy-paste commands for recording and replay ('AGENT_HOST_REPLAY_RECORD=1 ./scripts/test-integration.sh --run src/...'), a concrete gating pattern ('(RECORD ? test : test.skip)'), named files and functions ('capiReplayProxy.ts', 'hostOnlyTest(...)'), and pipeline definition '111'. The Azure queue workflow is parameterized but its actual command is deferred to 'The azure-pipelines skill contains the canonical command' — a minor executable gap that keeps it below fully copy-paste-ready.

4 / 5

Workflow Clarity

Each workflow (write-and-evaluate, record, gate, Azure validation) is a numbered sequence with explicit validation checkpoints and error-recovery loops: 'Run plain replay (no env var) to confirm green, then commit', 'If something leaked, the fix is to extend normalization/redaction... and re-record — not to edit the fixture', and rerun rules after a platform-specific failure. Verification steps are never implicit.

5 / 5

Progressive Disclosure

No bundle files exist; the skill appropriately delegates architecture and troubleshooting to a single, prominently signaled one-level-deep README ('Before doing anything, read the architecture + troubleshooting reference... This skill is only the workflows; the README is the source of truth for how it works'), and delegates CI-artifact and Azure-pipeline mechanics to sibling skills instead of duplicating them. Sections are cleanly divided per workflow.

5 / 5

Total

19

/

20

Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A model description: explicit 'Use when' triggers covering the full task lifecycle, five concrete capabilities with domain-specific vocabulary, symptom-based triggers for troubleshooting, and a hard scope anchor via the source path. It is also concise, with no padding or over-claims.

DimensionReasoningScore

Specificity

It lists five concrete actions — 'adding a cross-provider test', 're-recording fixtures after an SDK bump', 'required cross-platform Azure validation', 'gating non-deterministic or platform-specific tests', and 'diagnosing replay cache misses' — each with domain-specific objects. Coverage of the skill's workflows is comprehensive with no gaps, matching the top anchor rather than 'minor gaps in coverage'.

5 / 5

Completeness

It explicitly answers 'when' with a concrete 'Use when writing, recording, updating, validating, or troubleshooting...' clause and 'what' with 'Covers adding a cross-provider test, re-recording fixtures after an SDK bump, required cross-platform Azure validation, gating... and diagnosing replay cache misses'. Both halves are explicit with concrete trigger phrases, matching the anchor-5 example pattern.

5 / 5

Trigger Term Quality

Natural task-lifecycle phrases ('writing, recording, updating, validating, or troubleshooting... end-to-end and real-provider integration tests') are joined by symptom-based triggers ('diagnosing replay cache misses') and both 'end-to-end' and 'e2e' forms (the latter via the path 'test/node/e2e'). A developer needing this skill would naturally say these terms.

5 / 5

Distinctiveness Conflict Risk

The description is pinned to a niche location — 'Agent Host end-to-end and real-provider integration tests under src/vs/platform/agentHost/test/node/e2e and providerIntegration' — with workflow terms (fixtures, SDK bump, replay cache) that no neighboring skill would claim. Clear niche, minimal conflict risk.

5 / 5

Total

20

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

relative_links

Relative link issues: 1 suspicious

Warning

Total

15

/

16

Passed

Repository

microsoft/vscode

Path

.github/skills/agent-host-e2e-tests/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.