CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

ai-evals

Author and run black-box benchmark cases for the Windmill AI generation modes (flow/app/script/cli/global) in ai_evals/. Use when adding or changing eval cases, or when running before/after benchmarks for AI chat / copilot changes.

Quality

92%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

93%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A tight, highly operational skill: executable commands with environment prerequisites, concrete good/bad authoring examples, and clean delegation of the full case-format spec to ai_evals/README.md. The only minor gap is the absence of an explicit post-authoring verification step in the authoring workflow.

DimensionReasoningScore

Conciseness

The body is lean and dense with project-specific operational detail (workspace reuse limits, the separate judge model, global-mode workspace behavior) and never explains general concepts Claude already knows. Every section carries non-obvious information that earns its tokens.

5 / 5

Actionability

Commands are copy-paste ready (`bun run cli -- run global <caseIds...> --models sonnet,gpt-5.5,...` with the required env vars) and cover the common cases (install, list models, list cases, run). Good/bad examples for prompts, hard checks, and judge checklists give concrete authoring guidance.

5 / 5

Workflow Clarity

The running workflow is clearly sequenced (install → list → run with backend env setup) and the validation philosophy (narrow hard deterministic checks vs semantic judgeChecklist) is explicit. The authoring path lacks an explicit final checkpoint like 'run the new case to verify it passes', which keeps it at 4 rather than 5.

4 / 5

Progressive Disclosure

The body is a well-organized overview that delegates the full case format and fixture details to a clearly signaled one-level-deep reference ('See ai_evals/README.md for the full case format, fields, and fixture details') with no nested reference chains. No bundle files exist in references/, scripts/, or assets/, and nothing in the body pretends they do.

5 / 5

Total

19

/

20

Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description that states what the skill does and when to use it with concrete, domain-specific triggers, in third person and without padding. Minor room to broaden trigger synonyms and enumerate a couple more capabilities.

DimensionReasoningScore

Specificity

Names two concrete actions ('Author and run black-box benchmark cases'), the exact location ('in ai_evals/'), and enumerates all five generation modes (flow/app/script/cli/global). It stays at 4 rather than 5 because only two actions are listed rather than a comprehensive set, and well above 3 because the actions are concrete and precisely scoped.

4 / 5

Completeness

Explicitly answers both questions: the 'what' ('Author and run black-box benchmark cases for the Windmill AI generation modes... in ai_evals/') and the 'when' with two concrete trigger clauses ('Use when adding or changing eval cases, or when running before/after benchmarks for AI chat / copilot changes'). This matches the anchor-5 example structure exactly.

5 / 5

Trigger Term Quality

Natural trigger phrases users would actually say are present: 'adding or changing eval cases', 'running before/after benchmarks', 'AI chat / copilot changes'. A few natural synonyms (e.g., 'evaluations', 'prompt changes', 'regression testing') are missing, keeping it just below comprehensive coverage.

4 / 5

Distinctiveness Conflict Risk

The description is anchored to a distinct niche — Windmill's named AI generation modes and the ai_evals/ directory — so it would rarely fire for an unrelated skill. Third person is used throughout with no over-claims.

5 / 5

Total

18

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

windmill-labs/windmill

Path

.agents/skills/ai-evals/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.