serving-llms-vllm

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

Quality

78%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Critical

Do not install without reviewing

Fix and improve this skill with Tessl

tessl review fix ./12-inference-serving/vllm/SKILL.md

Quality

Content

57%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is well-structured and largely executable with strong progressive disclosure to real reference files, but loses points on a few placeholder steps and missing validation checkpoints in the batch workflow.

Suggestions

Replace the placeholder load-test step (workflow 1 Step 2) and accuracy-verify step (workflow 3 Step 4) with concrete executable commands or code instead of comment-only pseudocode.

Add an explicit validation/verification checkpoint to the batch-inference workflow (workflow 2) — e.g., confirm output count matches input count and spot-check a sample before declaring the batch done.

Tighten the repeated `vllm serve MODEL --flag` snippets in 'Common issues' by referencing a shared flags summary rather than restating each invocation.

Dimension	Reasoning	Score
Conciseness	Mostly efficient and free of basic-concept padding, but recurring `vllm serve MODEL --flag` blocks restate flags and a few steps (workflow 1 locust setup, workflow 3 accuracy check) are placeholder comments that add noise without adding value.	2 / 3
Actionability	The bulk is copy-paste ready (full Python snippets, real `vllm serve`/`docker run`/`curl` commands), but workflow 1 Step 2 (load test) and workflow 3 Step 4 (verify accuracy) ship only comment pseudocode, leaving key guidance incomplete.	2 / 3
Workflow Clarity	The three workflows are clearly sequenced with checklists and the production workflow has verification targets, but the batch-inference workflow (workflow 2) lacks any validation/verify checkpoint, which caps clarity at 2 per the batch-operations rule.	2 / 3
Progressive Disclosure	SKILL.md is a focused overview with a dedicated Advanced topics section pointing one level deep to four real, verified reference files (server-deployment.md, optimization.md, quantization.md, troubleshooting.md) via clear link signaling.	3 / 3
	Total	9 / 12 Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is concise yet specific, naming concrete capabilities with an explicit 'Use when' trigger and natural user keywords. It clearly occupies a distinct niche with low conflict risk.

Dimension	Reasoning	Score
Specificity	Lists multiple specific concrete capabilities — "Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching" and "Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism" — rather than vague domain naming.	3 / 3
Completeness	Explicitly answers both what ("Serves LLMs... Supports...") and when ("Use when deploying production LLM APIs...") with an explicit trigger clause, matching the top anchor.	3 / 3
Trigger Term Quality	Natural user phrasing with good coverage: "deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory" matches terms a user would actually say.	3 / 3
Distinctiveness Conflict Risk	The vLLM-specific niche (PagedAttention, named library, quantization types) and distinct triggers make it unlikely to fire for the wrong skill.	3 / 3
	Total	12 / 12 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: Orchestra-Research/AI-Research-SKILLs
Path: 12-inference-serving/vllm/SKILL.md
Commit: 773a529

Reviewed: about 5 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.