CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

rewardkit

Write Harbor task verifiers using Reward Kit. Use when creating or editing a task's tests/ directory, adding grading criteria, setting up LLM/agent judges, or designing verifiers that produce a reward score.

71

Quality

89%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

86%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

Excellent executable reference content with near-zero padding, but it is a monolithic single-file skill: the entire API surface lives in SKILL.md with no progressive disclosure via reference files, and no explicit local validation loop for iterating on a verifier.

Suggestions

Move the exhaustive built-ins list, the '[judge]' options paragraph, and the aggregation-mode semantics into a references/ file (e.g. references/api.md), keeping SKILL.md to setup, core patterns, and 'When to reach for what', so the full reference is loaded only when needed.

Add a short validation step showing how to run the verifier outside Harbor (e.g. 'uvx --from 'harbor-rewardkit==0.2.*' rewardkit /tests') and inspect reward-details.json to debug failing criteria before a real run.

Restructure the '[judge] options' paragraph as a bulleted list or table so each option (timeout, reasoning_effort, guard, isolated, samples) is scannable rather than buried in dense prose.

DimensionReasoningScore

Conciseness

Lean and dense throughout — no padding and no explanations of concepts Claude already knows; every sentence carries tool-specific facts ('Zero-parameter criteria auto-register', 'all-pass, any-pass, and required-pass count a score as passing only when it is 1.0', the network_mode baseline/override semantics). Not 4 because no instance of over-explanation was found to trim; the length is informational, not verbosity.

5 / 5

Actionability

Fully executable, copy-paste-ready examples cover the common cases: the 'uvx --from 'harbor-rewardkit==0.2.*' rewardkit /tests' test.sh, Python criteria with '@criterion' and parameterized calls, judge TOMLs for LLM/agent/JEV judges, and reward.toml aggregation snippets with expected JSON outputs. Not 4 because the examples are complete and executable as written, not just fragments with gaps.

5 / 5

Workflow Clarity

Sections form a coherent sequence (setup → programmatic → custom → judges → scoring → multi-reward → multi-step → selection guide) with most checkpoints present: output files are documented ('/logs/verifier/reward-details.json — per-criterion results, judge reasoning, errors') and risky behaviors are flagged (isolated=True to avoid corrupting the workspace). Not 5 because there is no explicit validation/feedback loop for locally running and debugging a verifier before a Harbor run; not 3 because the sequence and checkpoints are clearly better than implicit.

4 / 5

Progressive Disclosure

The body is well-sectioned with clear headers, but roughly 340 lines of exhaustive reference material — the full built-ins list, the dense '[judge]' options paragraph, all aggregation modes, and multi-step semantics — are fully inlined in SKILL.md with no bundle files at all (the only pointer is external: 'See examples/tasks/reward-kit-example/ in the Harbor repo'). This matches the anchor 'content that should be separate is inline'; not 4 because there are no well-signaled one-level-deep references to offset the inlined volume.

3 / 5

Total

17

/

20

Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: third-person voice, explicit 'what' and 'when', concrete trigger phrases, and a distinct niche. The only weaknesses are unmentioned synonyms and slightly incomplete coverage of the tool's capability surface.

DimensionReasoningScore

Specificity

Lists several concrete actions — 'creating or editing a task's tests/ directory, adding grading criteria, setting up LLM/agent judges, or designing verifiers that produce a reward score' — each a specific capability. Falls short of 5 because major capabilities from the body (programmatic built-in checks, scoring aggregation, multi-reward subdirectories) are not surfaced, leaving minor coverage gaps; well above 3, which expects only 1-2 concrete actions.

4 / 5

Completeness

Explicitly answers both: what — 'Write Harbor task verifiers using Reward Kit' — and when — 'Use when creating or editing a task's tests/ directory, adding grading criteria, setting up LLM/agent judges, or designing verifiers that produce a reward score.' The explicit 'Use when' clause with concrete trigger phrases matches the top anchor; not 4 because the 'when' is already fully explicit and specific.

5 / 5

Trigger Term Quality

Natural phrases users would say are present: 'task's tests/ directory', 'grading criteria', 'LLM/agent judges', 'verifiers that produce a reward score'. A few natural synonyms and variations ('reward function', 'rubric', 'test suite', 'score a task') are missing, so it matches 'good keyword coverage; a few natural terms missing' rather than the comprehensive synonym-level coverage of 5.

4 / 5

Distinctiveness Conflict Risk

The niche is clearly delimited by 'Harbor task verifiers' and 'Reward Kit' with triggers tied to 'tests/ directory' and verifier/judge setup — terms unlikely to appear when other skills are needed, giving minimal conflict risk.

5 / 5

Total

18

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

Arize-ai/phoenix

Path

.agents/skills/rewardkit/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.