Name: online-evals
Rating: 55.2 (1 reviews)
Author: launchdarkly

online-evals

Attach judges to config variations for automatic LLM-as-a-judge evaluation. Create custom judges, configure sampling rates, and monitor quality scores.

Quality

61%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

High

Do not use without reviewing

Fix and improve this skill with Tessl

tessl review fix ./skills/agentcontrol/online-evals/SKILL.md

Quality

Content

65%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is highly actionable with executable code throughout, but it is held back by inlined full implementations that belong in separate reference files, missing validation before the destructive judge-array replacement, and trimmable SDK boilerplate.

Suggestions

Move the full AIConfigJudges Python class and the two SDK examples into files under references/ and keep only concise snippets plus links in SKILL.md, improving both conciseness and progressive_disclosure.

Add an explicit validation step before the variation PATCH (e.g. GET current judgeConfiguration and confirm the intended judges) so the destructive replace-all behavior has a checkpoint, lifting workflow_clarity.

Trim SDK boilerplate (context building, flush/close ceremony) in the inline examples to the minimal lines that illustrate judge evaluation, or push the full ceremony into the referenced example file.

Dimension	Reasoning	Score
Conciseness	The body is mostly efficient with concrete API specifics, but the two full SDK async examples carry boilerplate (Context building with 'Sandy', full event-flush/close ceremony) that could be trimmed, fitting the score-3 anchor of "mostly efficient but could be tightened."	3 / 5
Actionability	Provides fully executable, copy-paste-ready curl commands with complete JSON payloads, a working Python class, and complete SDK examples covering the common cases, matching the score-5 anchor.	5 / 5
Workflow Clarity	Steps 1-3 are sequenced, but the destructive "judges array replaces all existing judge attachments" operation has a warning yet no explicit verification checkpoint (e.g. fetch current judges first), so per the feedback-loop cap workflow_clarity cannot exceed 3.	3 / 5
Progressive Disclosure	There is good section structure and clearly signaled external references, but the full Python class (~130 lines) and two full SDK examples are inlined rather than split into separate files, matching the score-3 anchor of "some structure but content that should be separate is inline."	3 / 5
	Total	14 / 20 Passed

Description

58%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description clearly communicates what the skill does with several concrete actions, but it omits any explicit "Use when..." trigger guidance, which caps completeness at 3 and leaves trigger-term coverage at relevant-but-incomplete.

Suggestions

Add an explicit trigger clause, e.g. "Use when evaluating AI config responses with LLM-as-a-judge, attaching judges to variations, or monitoring quality scores."

Broaden trigger terms with natural synonyms and concrete artifacts users mention (e.g. "AI configs", "eval metrics", "accuracy/relevance/toxicity judges", "sampling rate") to lift trigger_term_quality toward 4-5.

Tighten the distinctiveness phrasing to reference the LaunchDarkly-specific judge mode explicitly, reducing overlap with generic evaluation skills.

Dimension	Reasoning	Score
Specificity	Lists several concrete actions — "Attach judges to config variations", "Create custom judges", "configure sampling rates", and "monitor quality scores" — with only minor gaps in coverage, matching the score-4 anchor rather than the comprehensive score-5 anchor.	4 / 5
Completeness	The "what" is clearly stated but there is no "Use when..." clause or equivalent explicit trigger guidance, so per the judging guidelines completeness is capped at 3; it is not 4 because "when" is entirely absent rather than weakly present.	3 / 5
Trigger Term Quality	Contains relevant domain keywords ("judges", "config variations", "LLM-as-a-judge", "sampling rates", "quality scores") but lacks the natural trigger phrases a user would say and has no synonyms, fitting the score-3 anchor of "some relevant keywords but missing common variations."	3 / 5
Distinctiveness Conflict Risk	The niche (LLM-as-a-judge evaluation of LaunchDarkly AI config variations) is mostly distinct with only minor overlap risk against related config skills, fitting score 4 rather than the clearly-niched score 5 because triggers are not fully spelled out.	4 / 5
	Total	14 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: launchdarkly/ai-tooling
Path: skills/agentcontrol/online-evals/SKILL.md
Commit: 82ce1ba

Reviewed: 9 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.