online-evals

Attach judges to config variations for automatic LLM-as-a-judge evaluation. Create custom judges, configure sampling rates, and monitor quality scores.

Quality

79%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

High

Do not use without reviewing

Fix and improve this skill with Tessl

tessl review fix ./skills/agentcontrol/online-evals/SKILL.md

Quality

Content

82%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A dense, highly actionable skill body with executable API and SDK examples, a clear sequenced workflow, and useful error-handling guidance. Its main weaknesses are mild redundancy across the fallthrough and SDK sections and inline code that could be split into bundle files for better progressive disclosure.

Dimension	Reasoning	Score
Conciseness	The body is operational throughout — curl commands, tables, and code with almost no explanation of concepts Claude already knows. It misses anchor 5 due to repetition: the fallthrough caveat appears in Step 3, the Python docstrings, and Next Steps, and the two SDK examples duplicate setup boilerplate; it is not anchor 3 because the redundancy is trimmingmable rather than unnecessary explanation.	4 / 5
Actionability	Fully executable, copy-paste-ready guidance: complete curl requests with headers and payloads, a working Python class covering create/attach/fallthrough, two complete SDK examples, and a status-code error table mapping cause to solution. Specific examples cover both automatic and direct evaluation paths.	5 / 5
Workflow Clarity	Clear Step 1→2→3 sequence with an important warning that the judges array replaces existing attachments, an error-recovery table, and a Next Steps checklist. Minor gap: no explicit verification that a judge config is enabled (fallthrough set) before attaching it, keeping it below anchor 5; checkpoints otherwise present so above anchor 3.	4 / 5
Progressive Disclosure	Good structure with clear sections and a well-signaled, one-level-deep References section pointing to official docs and SDK examples. No bundle files exist, and the ~127-line inline Python class and near-duplicate SDK examples are content that could live in scripts/references files, preventing a 5; organization is too solid for a 3.	4 / 5
	Total	17 / 20 Passed

Description

66%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A specific, third-person description naming concrete capabilities in a distinct niche, but it lacks any 'Use when...' trigger guidance, capping its completeness and weakening its discoverability. Adding an explicit trigger clause with natural synonyms would lift it substantially.

Suggestions

Append an explicit trigger clause, e.g., 'Use when the user wants to score or evaluate AI config responses, set up LLM-as-a-judge quality checks, or monitor evaluation metrics.'

Add natural synonyms users might say, such as 'score responses', 'grade outputs', or 'quality evaluation', to improve trigger term coverage.

Disambiguate from the sibling configs-create skill by clarifying that this skill is for attaching/monitoring judges rather than creating base configs.

Dimension	Reasoning	Score
Specificity	Lists several concrete actions ('Attach judges to config variations', 'Create custom judges', 'configure sampling rates', 'monitor quality scores') with only minor gaps such as direct/SDK evaluation and guardrail integration. It falls short of anchor 5's comprehensive coverage but exceeds anchor 3's '1-2 concrete actions'.	4 / 5
Completeness	The 'what' is clear and multi-action, but there is no 'Use when...' clause or equivalent explicit trigger guidance, which caps completeness at 3 per the judging guidelines. Not a 2 because the 'what' is concrete, not vague; not a 4 because 'when' is entirely absent rather than just imprecise.	3 / 5
Trigger Term Quality	Includes good natural keywords users would say ('judges', 'LLM-as-a-judge evaluation', 'quality scores', 'sampling rates'). A few natural terms are missing (e.g., 'score responses', 'grade outputs', 'AI config evaluation'), placing it between anchor 4 and 5 but noticeably above the midpoint.	4 / 5
Distinctiveness Conflict Risk	Occupies a clear niche (LLM-as-a-judge evaluation of AI configs) with distinct triggers like 'judges' and 'evaluation scores'. Minor overlap risk with the sibling skill 'configs-create', which also mentions creating judges, keeping it below anchor 5.	4 / 5
	Total	15 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: launchdarkly/ai-tooling
Path: skills/agentcontrol/online-evals/SKILL.md
Commit: ef54971

Reviewed: 6 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.