CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

serving-llms-vllm

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism. Also covers llama.cpp and SGLang (guides in references/).

Quality

84%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

SKILL.md
Quality
Evals
Security

Quality

Content

68%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is well-structured and largely executable — concrete vllm serve commands, Docker deployment, monitoring checks, and troubleshooting fixes — with appropriate offloading of depth to reference files. Main gaps: the batch-inference workflow lacks validation (capping workflow clarity), several steps are comment-only stubs, and the merged sub-skill guides introduce two-level-deep references.

Suggestions

Add validation to Workflow 2 (batch inference): after generation, verify len(results) matches len(prompts), check for empty/failed outputs, and handle errors before writing results.jsonl — batch operations require verification checkpoints per the rubric.

Replace comment-only code blocks with executable content: a minimal locust test_load.py example for the load-test step, a concrete HF search command or model ID for finding AWQ models, and a runnable snippet comparing quantized vs. baseline outputs.

Flatten or signpost the second-level references: list the sub-guides under references/llama-cpp/references/ and references/sglang/references/ in the related-tools table (or restructure to one level deep) so all detailed content is discoverable directly from SKILL.md.

DimensionReasoningScore

Conciseness

The body is dominated by commands and code with one-line explanations ("vLLM achieves 24x higher throughput than standard transformers through PagedAttention (block-based KV cache)"); little of what Claude already knows is re-explained. Not score 5: there are trimmable elements — three comment-only code blocks ("# Create test_load.py with sample requests", "# Compare quantized vs non-quantized responses") and repeated 'Copy this checklist' boilerplate. Not score 3: these are minor instances, not sections of unnecessary explanation.

4 / 5

Actionability

Most guidance is copy-paste ready: "vllm serve meta-llama/Llama-3-8B-Instruct --gpu-memory-utilization 0.9", a Docker run command, a full offline-inference Python script, and metric-check curls. Not score 5: several steps are comment-only stubs rather than executable content — the locust load test ("# Create test_load.py with sample requests"), quantized-model discovery ("# Search for AWQ models"), and accuracy verification ("# Compare quantized vs non-quantized responses") leave gaps in the common workflows.

4 / 5

Workflow Clarity

Workflow 1 has real checkpoints ("Verify TTFT < 500ms and throughput > 100 req/sec", Step 5's metrics list), but Workflow 2 — explicitly a batch operation ("Offline batch inference... processing large datasets") — has no validation step (no check that output count matches input, no handling of failed generations), and Workflow 3's "Step 4: Verify accuracy" is a comment-only stub with no method. The rubric caps batch workflows lacking validation at 3, which takes precedence over Workflow 1's otherwise good sequencing. Not score 4: the batch-operation validation cap applies.

3 / 5

Progressive Disclosure

SKILL.md is a genuine overview: quick start, workflows, and an Advanced topics section with well-signaled one-line pointers ("See [references/server-deployment.md]... for Docker, Kubernetes, and load balancing configurations"), plus a related-tools table describing each sub-guide; all referenced paths exist in the bundle. Not score 5: the merged sub-skills add a second reference level (llama-cpp/overview.md → llama-cpp/references/server.md; sglang/overview.md → sglang/references/deployment.md), so references are not uniformly one level deep. Not score 3: the split is appropriate and navigation from the top level is easy.

4 / 5

Total

15

/

20

Passed

Description

96%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: third-person voice, concrete capabilities with named technologies, and an explicit 'Use when' clause with multiple natural trigger phrases covering deployment, optimization, and memory-constrained serving. The only weakness is that claiming coverage of llama.cpp and SGLang slightly broadens conflict risk with dedicated per-tool skills.

DimensionReasoningScore

Specificity

"Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching", "Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism" — multiple specific concrete actions (serving, endpoints, named quantization methods, parallelism) with comprehensive coverage of the serving domain. Not score 4: coverage spans serving, deployment, optimization, and memory-constrained scenarios with no meaningful gaps.

5 / 5

Completeness

Explicitly answers 'what' ("Serves LLMs with high throughput... Supports OpenAI-compatible endpoints, quantization... and tensor parallelism") and 'when' ("Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory") with concrete trigger phrases, matching the anchor-5 example pattern. Neither component is missing or vague.

5 / 5

Trigger Term Quality

Natural phrases users would say include "deploying production LLM APIs", "optimizing inference latency/throughput", "serving models with limited GPU memory", plus explicit tool names (vLLM, llama.cpp, SGLang) that are the most common natural triggers. Not score 4: synonyms across usage scenarios (deploy/serve/optimize) and tool names give comprehensive coverage; only trivial variants like "self-hosting" are absent.

5 / 5

Distinctiveness Conflict Risk

"vLLM's PagedAttention and continuous batching" plus named quantization methods carve a clear serving niche, but "Also covers llama.cpp and SGLang" creates minor overlap risk with hypothetical dedicated skills for those tools (e.g., a standalone llama.cpp skill covering Apple Silicon/edge inference). Not score 5: the explicit secondary coverage of two other engines widens the trigger surface beyond a single distinct niche.

4 / 5

Total

19

/

20

Passed

Validation

81%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 13 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

relative_links

Relative link issues: 2 deeper-than-1-level

Warning

referenced_paths_exist

Referenced path issues: 4 deeper-than-1-level

Warning

Total

13

/

16

Passed

Repository

OpenLAIR/dr-claw

Path

skills/inference-serving/vllm/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.