jetson-llm-benchmark

Benchmark Jetson LLM/VLM serving performance across vLLM, llama.cpp, and Ollama with structured JSON output.

Quality

74%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./skills/jetson-llm-benchmark/SKILL.md

Quality

Content

78%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is executable and well-structured with concrete commands, an explicit JSON output contract, and strong validation/error-handling feedback loops. The main opportunity is trimming minor redundancy and possibly moving the lengthy output-contract schemas into a reference file.

Suggestions

De-duplicate the vLLM container-tag explanation, which appears in both the vLLM path section and the Limitations section.

Consider moving the per-runtime JSON output-contract blocks into a references/ file (e.g., OUTPUT_CONTRACT.md) referenced once from the body to reduce inline length.

Consolidate the script list, which currently appears in both the "Available Scripts" table and the "Three paths" prose.

Dimension	Reasoning	Score
Conciseness	The body is dense and explicitly assumes Claude's competence ("LLMs already know what TTFT/ITL/throughput mean"), adding only Jetson-specific deltas; minor redundancy exists where the vLLM container tag and script lists are restated across the table, instructions, and limitations.	4 / 5
Actionability	Copy-paste-ready bash commands with concrete flags are given for all three runtimes, plus a "--help first" rule and exact JSON output shapes with field names — fully executable guidance covering the common cases.	5 / 5
Workflow Clarity	The workflow is clearly sequenced (pick runtime → --help → run → parse JSON → flag warnings → hand off) with explicit validation checkpoints (required warmup pass, warnings on bad power mode / background GPU / throttling) and exit-code feedback loops (2, 3) with remediation.	5 / 5
Progressive Disclosure	Content is organized into clear sections with one-level-deep references to real verified script files (scripts/bench_vllm.sh, bench_llama_cpp.sh, bench_ollama.sh) and external URLs; the long inline JSON output-contract block could arguably live in a separate reference file.	4 / 5
	Total	18 / 20 Passed

Description

70%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific and distinct, naming the three runtimes and the JSON output contract, but it omits any explicit "Use when..." trigger guidance, which limits its completeness. Adding a when-to-use clause would move it toward the top anchors.

Suggestions

Append a "Use when..." clause naming concrete triggers (e.g., "Use when benchmarking or comparing Jetson LLM/VLM serving latency and throughput across vLLM, llama.cpp, or Ollama.").

Add natural synonyms users say ("measure", "latency", "throughput", "compare runtimes") to broaden trigger-term coverage.

Optionally name the second concrete action beyond benchmarking (e.g., "compare runs via structured JSON") to strengthen specificity.

Dimension	Reasoning	Score
Specificity	"Benchmark Jetson LLM/VLM serving performance across vLLM, llama.cpp, and Ollama with structured JSON output" names the domain plus three concrete runtimes and a concrete output format — several specific capabilities, though it is a single verb across targets rather than multiple distinct actions.	4 / 5
Completeness	The description has a clear "what" (benchmark serving performance across three runtimes with JSON output) but no "Use when..." clause or equivalent explicit trigger guidance, which per the guidelines caps completeness at 3.	3 / 5
Trigger Term Quality	Natural trigger terms are present ("Benchmark", "LLM/VLM serving performance", "vLLM", "llama.cpp", "Ollama", "structured JSON") and match what a user would say, but common synonyms like "measure" or "throughput/latency" are not included.	4 / 5
Distinctiveness Conflict Risk	"Jetson LLM/VLM serving performance across vLLM, llama.cpp, and Ollama" carves a clear niche with distinct triggers and minimal overlap risk with other skills.	5 / 5
	Total	16 / 20 Passed

Validation

81%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 13 / 16 Passed

Validation for skill structure

Criteria	Description	Result
metadata_version	'metadata.version' is missing	Warning
metadata_field	'metadata' should map string keys to string values	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	13 / 16 Passed

Repository: NVIDIA/skills
Path: skills/jetson-llm-benchmark/SKILL.md
Commit: c3168ca

Reviewed: 2 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.