llm-judge

AI quality judge that scores agent responses 0-10 across helpfulness, accuracy, completeness, and clarity. Use when evaluating multi-agent output or implementing LLM-as-judge quality gates.

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

85%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is an efficient, well-structured instruction skill with a concrete output contract and a clear single-action workflow. Its main weakness is mild redundancy in the guardrail score bands and qualitative rather than precise dimension criteria with no worked example.

Suggestions

Remove or de-duplicate the guardrail score-band ranges (e.g. 'Score 3-5 for partial', 'Score 6-8 for solid') since they overlap with the dimension definitions, or fold them into the dimension list.

Tighten the per-dimension criteria from open questions into concrete rubric anchors (e.g. what 'Helpfulness' looks like at score 8 vs 5) so scoring is more reproducible.

Add one short worked example showing an input response and the expected JSON output to anchor the scoring scale.

Dimension	Reasoning	Score
Conciseness	The body is lean with compact sections and an exact output contract, though the Guardrails section restates scoring bands (3-5, 6-8) that partially duplicate the dimension descriptions, a minor instance of over-explanation.	4 / 5
Actionability	Provides a concrete, copy-paste-ready JSON output format with an explicit integer range and named dimensions, but the per-dimension criteria are qualitative questions rather than precise rubrics, and no worked scoring example is given.	4 / 5
Workflow Clarity	As a simple single-purpose skill, the action (score four dimensions, emit one JSON object) is unambiguous with a clear output contract; no destructive or batch operations are involved so the validation cap does not apply.	5 / 5
Progressive Disclosure	The skill is under 50 lines, has no bundle files, and no external references are needed; content is organized into clearly headed sections (Skills, Output Format, Guardrails), satisfying the simple-skill exception for progressive disclosure.	5 / 5
	Total	18 / 20 Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific and complete, clearly stating what the skill does and when to use it with concrete trigger phrases. It is distinctive with low conflict risk; the only minor gap is a slightly narrower action vocabulary that keeps specificity just short of the top anchor.

Dimension	Reasoning	Score
Specificity	Lists the domain (scoring agent responses) plus four concrete scoring dimensions (helpfulness, accuracy, completeness, clarity), giving several specific concrete aspects, but remains a single parameterized action rather than multiple distinct actions.	4 / 5
Completeness	Explicitly answers both 'what' (scores responses 0-10 across four named dimensions) and 'when' (a concrete 'Use when evaluating multi-agent output or implementing LLM-as-judge quality gates' clause).	5 / 5
Trigger Term Quality	Includes natural user-facing phrases like 'evaluating multi-agent output' and 'implementing LLM-as-judge quality gates', with good keyword coverage, though a few natural synonyms (e.g. 'grading', 'scoring') are absent.	4 / 5
Distinctiveness Conflict Risk	Targets a clear niche ('AI quality judge', 'LLM-as-judge', 'multi-agent output') with distinct triggers and minimal overlap risk against unrelated skills.	5 / 5
	Total	18 / 20 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
metadata_version	'metadata.version' is missing	Warning
metadata_field	'metadata' should map string keys to string values	Warning

	Total	14 / 16 Passed

Repository: Atmosphere/atmosphere
Path: modules/skills/src/main/resources/META-INF/skills/llm-judge/SKILL.md
Commit: b600b8a

Reviewed: 2 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.