CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

rewardkit

Write Harbor task verifiers using Reward Kit. Use when creating or editing a task's tests/ directory, adding grading criteria, setting up LLM/agent judges, or designing verifiers that produce a reward score.

Quality

90%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

82%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A high-quality, reference-style skill body: nearly every token conveys Reward-Kit-specific detail Claude cannot know, with executable examples for every feature area and clear decision guidance. The main improvement opportunities are tightening the densest prose passages (environment/network modes, judge options) and moving the built-ins API catalog and [judge] options into a reference file to keep SKILL.md a leaner overview.

Suggestions

Move the full built-ins API listing and the "Useful [judge] options" catalog into a references/ file (e.g., references/builtins.md and references/judge-options.md), keeping SKILL.md as a lean overview with one-level-deep pointers.

Add an explicit validation checkpoint: "Run `uvx --from 'harbor-rewardkit==0.2.*' rewardkit /tests` locally and inspect /logs/verifier/reward-details.json for errors before finalizing the task" as a step near the top of the workflow.

Tighten the shared-vs-separate environment section — the phase-override vs. baseline distinction spans three paragraphs of intertwined conditionals that could be reduced to a short decision table.

DimensionReasoningScore

Conciseness

The body is dense and almost entirely novel API detail Claude cannot know (built-ins, TOML schemas, aggregation modes), with no filler explaining known concepts. It is not a 5 because a few passages could be trimmed or tightened — notably the shared-vs-separate environment paragraphs ("Put `[verifier].network_mode` only when verify() needs different network access... (a phase override, not a baseline)") and the single-paragraph wall of "Useful `[judge]` options" — but these are minor instances, matching the 4 anchor.

4 / 5

Actionability

Fully executable, copy-paste-ready guidance throughout: the exact `uvx --python '>=3.12,<3.14' --from 'harbor-rewardkit==0.2.*' rewardkit /tests` invocation, runnable Python and TOML snippets, directory layouts, and concrete output JSON. Specific examples cover the common cases (built-ins, custom criteria, LLM/agent/JEV judges, multi-reward aggregation). It is not a 4 because there are no gaps — even edge configuration (extras, env keys, CA certificates) is given concretely.

5 / 5

Workflow Clarity

Sections follow the natural authoring sequence — setup (test.sh, environments), then criteria types, then judges, then multi-reward/multi-step structure, then output files — with a "When to reach for what" decision guide. Validation feedback is partially present (reward-details.json exposes per-criterion results and errors), but there is no explicit "run the verifier locally and check reward-details.json before shipping" checkpoint loop, which keeps it below 5; the sequence and most checkpoints match the 4 anchor.

4 / 5

Progressive Disclosure

A single-file skill with no bundle, but well organized: clear hierarchical headers, an early worked example, a decision guide, and one clearly signaled external pointer ("See `examples/tasks/reward-kit-example/` in the Harbor repo"). It is not a 5 because at ~340 lines some reference-grade material (the full built-ins API list, the `[judge]` options catalog) is inlined in SKILL.md where a references/ file would keep the overview lean; it is not a 3 because structure and navigation are genuinely good, not merely present.

4 / 5

Total

17

/

20

Passed

Description

92%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: it clearly states what the skill does and gives an explicit, multi-trigger 'Use when' clause with concrete, natural phrases. The only weakness is moderate keyword breadth — synonyms like 'scoring', 'rubric', or 'evaluate' are absent.

DimensionReasoningScore

Specificity

The description lists multiple concrete actions — "creating or editing a task's tests/ directory", "adding grading criteria", "setting up LLM/agent judges", "designing verifiers that produce a reward score" — comprehensively covering the skill's main use cases. It is not a 4 because coverage of the distinct workflows (programmatic checks, judges, scoring) is complete for a description-length summary rather than having minor gaps.

5 / 5

Completeness

It explicitly answers both parts: what ("Write Harbor task verifiers using Reward Kit") and when ("Use when creating or editing a task's tests/ directory, adding grading criteria, setting up LLM/agent judges, or designing verifiers that produce a reward score") with concrete trigger phrases. This matches the anchor-5 example structure exactly; a 4 would require the 'when' clause to be less explicit or specific, which it is not.

5 / 5

Trigger Term Quality

Good natural keyword coverage: "task verifiers", "tests/ directory", "grading criteria", "LLM/agent judges", "reward score" — phrases a Harbor task author would plausibly say. It falls short of 5 because common synonyms and variations are missing (e.g., "scoring", "rubric", "evaluating", "test.sh"), whereas the 5 anchor expects comprehensive synonym/extension coverage.

4 / 5

Distinctiveness Conflict Risk

"Harbor task verifiers" and "Reward Kit" name a clear niche with distinct triggers (tests/ directory, grading criteria, judges) unlikely to fire for unrelated skills. It is not a 4 because the domain framing (Harbor/Reward Kit) leaves virtually no overlap risk with generic testing or document skills.

5 / 5

Total

19

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

Arize-ai/phoenix

Path

.agents/skills/rewardkit/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.