CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

run-model-eval

Test a local model server that is already running (Ollama, LM Studio, llama.cpp, MLX, vLLM, SGLang) for speed and practical capability, and dig into anything that looks off.

Quality

87%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

100%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

An exemplary skill body: fully executable commands at every step, tightly written with no concept explanations, explicit validation gates and feedback loops at each risky point, and a clean split between the manual and the tool/pack files it drives. The only unverifiable aspect is the existence of the referenced bundle files, which is a limitation of the provided extraction rather than a visible defect.

DimensionReasoningScore

Conciseness

The body is purely operational — commands, flags, JSON field names, and a failure taxonomy — with zero explanation of concepts Claude already knows ("A pass rate alone is not a finding"). The densest passages (pack task enumerations, the full/lean split) are decision-relevant facts rather than padding, so every token earns its place.

5 / 5

Actionability

Every stage has a copy-paste-ready command with exact flags ("probe --base-url <url> --model <id> --out <run_dir>", "run ... --quick --samples 1"), and all placeholders are defined in earlier steps. Failure reading names exact fields ("smoke.ok", "loaded[].context_length", "finish_reason", "reason"), and common cases like quantization comparison get exact commands (--label into the same run directory).

5 / 5

Workflow Clarity

Eight numbered stages from discovery to handoff with an explicit validation gate ("`smoke.ok` must be true. If not, report the error; do not continue"), progress monitoring (terminal sessions, the "DONE" log line), an error-diagnosis loop (section 6's named causes), and a targeted-experiment feedback loop (section 7). Risky points also carry safety checkpoints, such as reading a vendor pack's unit tests before model-written code executes locally.

5 / 5

Progressive Disclosure

SKILL.md is a lean operating manual whose implementation and data are appropriately split into one-level-deep, clearly signaled files ("holaeval.py in this skill's directory", "packs/general-v1.json", "packs/agent-v1.json"), with nothing reference-sized inlined and trivial navigation via numbered sections. Note: the referenced tool/pack files are not present in this review bundle so their existence could not be verified, but the structure and signaling are exemplary.

5 / 5

Total

20

/

20

Passed

Description

70%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, distinctive description with excellent natural trigger terms via the engine names, but it omits any explicit 'Use when...' trigger guidance, which both caps completeness and leaves the common terms 'benchmark'/'eval' unused. Adding a trigger clause with user-phrase examples would raise it to top level.

Suggestions

Add an explicit trigger clause, e.g.: 'Use when the user asks to test, benchmark, or compare a model running on their own machine or LAN (e.g., "how fast is my LM Studio model", "Q4 vs Q8").'

Include the natural terms 'benchmark' and 'eval' in the description — they are among the most common ways users phrase this request and are currently absent.

Replace the abstract 'practical capability' with the concrete capabilities tested (e.g., 'math, coding, tool calling, and long-context tasks') to sharpen the what.

DimensionReasoningScore

Specificity

"Test a local model server ... for speed and practical capability, and dig into anything that looks off" names several concrete actions (speed testing, capability testing, anomaly investigation) plus six named engines, but "practical capability" is abstract and benchmarking/comparison is not explicitly named. It exceeds anchor 3's 1-2 concrete actions but falls short of anchor 5's fully enumerated, comprehensive action list.

4 / 5

Completeness

The description clearly answers "what" (tests a running local server for speed and practical capability, digs into anomalies) but contains no "Use when..." clause or equivalent explicit trigger guidance; "that is already running" states a precondition, not a when. Per the rubric guideline, a missing 'Use when' clause caps completeness at 3.

3 / 5

Trigger Term Quality

The engine names ("Ollama, LM Studio, llama.cpp, MLX, vLLM, SGLang") plus "local model server" and "speed" are terms users naturally say when they need this skill. Not anchor 5: the most natural phrasings for this request — "benchmark" and "eval" — are missing, so coverage is good but not comprehensive.

4 / 5

Distinctiveness Conflict Risk

"local model server that is already running" combined with six named engines carves a clear niche with distinct triggers; few skills would overlap and wrong-skill activation risk is minimal. No realistic closely-related-skill overlap that would justify anchor 4.

5 / 5

Total

16

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

holaboss-ai/holaOS

Path

runtime/api-server/src/embedded-capabilities/local-model-review/skills/run-model-eval/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.