online-evals

Attach judges to config variations for automatic LLM-as-a-judge evaluation. Create custom judges, configure sampling rates, and monitor quality scores.

Quality

66%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

High

Do not use without reviewing

Fix and improve this skill with Tessl

tessl review fix ./skills/agentcontrol/online-evals/SKILL.md

Quality

Content

65%Weight 40%Scale 1-3

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable with complete code and a clear sequenced workflow, but it is monolithic and somewhat redundant, with large inline implementation blocks that could be split out and no explicit verification step after destructive judge-attachment operations.

Suggestions

Move the full Python Implementation and SDK example blocks into separate reference files (e.g. scripts/ or references/) and link to them from a concise overview, reducing the inline token load.

Add an explicit verification step after Step 2 (e.g. GET the variation and confirm judgeConfiguration reflects the intended judges) to close the validation gap on the destructive replace-all PATCH.

Drop or condense the duplicate Python wrapper for the curl calls, keeping only one canonical form per operation to tighten conciseness.

Dimension	Reasoning	Score
Conciseness	Most content is concrete API detail and executable code, but the full Python class largely duplicates the preceding curl calls and the conceptual sections add length that could be trimmed, placing it at the score-2 'mostly efficient but could be tightened' anchor rather than 3.	2 / 3
Actionability	Provides complete, executable curl payloads, a full Python manager class, and working SDK snippets that are copy-paste ready, matching the score-3 anchor for fully executable guidance.	3 / 3
Workflow Clarity	The three-step workflow is clearly sequenced with gotcha notes and an error-handling table, but there is no explicit verification checkpoint after the destructive 'judges array replaces all existing' PATCH, so it does not reach the score-3 explicit-validation anchor.	2 / 3
Progressive Disclosure	Sections are well-organized with clearly signaled external references, but the 447-line body is monolithic — the large Python Implementation and SDK blocks are inline rather than split into one-level-deep bundle files, matching the score-2 anchor where content that should be separate is inline.	2 / 3
	Total	9 / 12 Passed

Description

67%Weight 40%Scale 1-3

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific and clearly occupies a distinct niche, but it omits an explicit 'Use when...' trigger clause and its keywords lean somewhat technical, which limits trigger-term quality and completeness.

Suggestions

Add an explicit 'Use when...' clause, e.g. 'Use when setting up automatic quality scoring for AI config variations, or when the user mentions judges, LLM-as-a-judge evaluation, sampling rates, or quality scores.'

Broaden trigger terms with natural user phrasings such as 'evaluate AI responses', 'quality monitoring', and 'judge scoring' alongside the existing technical terms.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions — 'Attach judges to config variations', 'Create custom judges', 'configure sampling rates', and 'monitor quality scores' — matching the score-3 anchor that expects several specific actions.	3 / 3
Completeness	The 'what' is clearly stated, but there is no 'Use when...' clause or equivalent explicit trigger guidance, which per the rubric guidelines caps completeness at 2 rather than 3.	2 / 3
Trigger Term Quality	'judges', 'sampling rates', and 'quality scores' are natural terms, but 'config variations' and 'LLM-as-a-judge evaluation' lean technical and common user phrasings are not fully covered, so it sits at the score-2 anchor rather than 3.	2 / 3
Distinctiveness Conflict Risk	It targets a clear niche — LaunchDarkly AI-config judges — with distinct triggers, making it unlikely to fire for an unrelated skill, matching the score-3 anchor.	3 / 3
	Total	10 / 12 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: launchdarkly/ai-tooling
Path: skills/agentcontrol/online-evals/SKILL.md
Commit: add614f

Reviewed: 6 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.