CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

arize-prompt-optimization

Optimizes, improves, and debugs LLM prompts using production trace data, evaluations, and annotations. Extracts prompts from spans, gathers performance signal, and runs a data-driven optimization loop using the ax CLI. Use when the user mentions optimize prompt, improve prompt, make AI respond better, improve output quality, prompt engineering, prompt tuning, or system prompt improvement.

Quality

94%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

SKILL.md
Quality
Evals
Security

Quality

Content

88%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A strong, highly actionable body: the phased workflow is explicit with real validation checkpoints and a genuine iterate/measure feedback loop, and almost all guidance is executable as written. The main costs are token weight from the inlined meta-prompt template and a generic best-practices section Claude largely already knows.

Suggestions

Move the 70-line optimization meta-prompt template (Phase 3) to a bundle file such as references/optimization-metaprompt.md and keep a 3-4 line summary with the placeholder list in SKILL.md, cutting roughly a sixth of the always-loaded context.

Drop or drastically trim the 'Prompt Engineering Best Practices' table — personas, delimiters, few-shot examples, and reasoning instructions are knowledge Claude already has; keep only the Arize-specific rows such as variable preservation ({var} vs {{var}}).

Show the full `ax experiments create` invocation (with dataset, prompt, and space arguments) in Phase 4 or the Iterate workflow instead of the elided `ax experiments create ...`, since running an experiment is the one step of the loop without a copy-paste command.

DimensionReasoningScore

Conciseness

The body is dense and command-driven (ax CLI invocations, jq scripts, tables of span attributes) with little padding, but two sections could be trimmed: the 'Prompt Engineering Best Practices' table restates techniques Claude already knows (personas, delimiters, few-shot examples, reasoning instructions), and the Workflows section repeats commands already shown in Phases 1-3. This fits 'Efficient; minor instances of over-explanation that could be trimmed' — not level 5, since not every token earns its place, and not level 3, since the excess is limited rather than an unnecessary explanation woven into the operational content.

4 / 5

Actionability

Nearly everything is copy-paste ready: complete `ax spans export PROJECT --filter ... -l 10 --stdout` commands, full jq join/select scripts with exact attribute paths, a complete fill-in-the-blanks optimization meta-prompt, and a troubleshooting table mapping exact errors to solutions. The common cases (failing trace, dataset+experiment, wrong format, RAG hallucination) each get worked examples. This matches 'Fully executable; copy-paste ready code or commands; specific examples cover the common cases'; the only elision is `ax experiments create ...`, which is minor relative to the overall coverage.

5 / 5

Workflow Clarity

The four phases are clearly sequenced (Extract → Gather → Optimize → Iterate), and validation checkpoints are explicit: after optimization, 'Verify all template variables are preserved', 'Check that format instructions are intact', 'Test on a few examples before full deployment'; Phase 4 closes the loop with failure analysis, score comparison, flipped-example counting, and regression checks ('Check for regressions -- examples that passed with prompt A but fail with prompt B'). The iterate loop plus troubleshooting table provides the error-recovery feedback loop the level-5 anchor requires; level 4 would require some of these checkpoints to be missing or implicit, which they are not.

5 / 5

Progressive Disclosure

References are real, one level deep, and clearly signaled ('see references/ax-setup.md', 'follow references/ax-profiles.md' — both files exist in the bundle, and there are no scripts/ or assets/ directories), and section headers make navigation easy. However, ~450 lines is large for SKILL.md: the 70-line meta-prompt template and the generic best-practices table are inlined content that arguably belongs in reference files, and no such split is offered. This fits 'Good structure; most content is appropriately placed; references mostly clear; minor organization gaps' rather than level 5's 'content appropriately split'.

4 / 5

Total

18

/

20

Passed

Description

96%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

An excellent description: third-person voice, concrete capability list, an explicit 'Use when...' clause with abundant natural trigger phrases, and a clearly signaled Arize-specific niche. The only weakness is mild trigger genericism ('make AI respond better', 'improve output quality') that creates minor overlap risk with adjacent skills.

DimensionReasoningScore

Specificity

The description lists multiple concrete, specific actions — "Optimizes, improves, and debugs LLM prompts", "Extracts prompts from spans", "gathers performance signal", "runs a data-driven optimization loop using the ax CLI" — together with the mechanisms (production trace data, evaluations, annotations). Coverage of the skill's capabilities is comprehensive, matching the anchor 'Lists multiple specific concrete actions; comprehensive coverage'; it exceeds the level-4 anchor, which requires minor gaps in coverage.

5 / 5

Completeness

The 'what' is explicit ("Optimizes, improves, and debugs LLM prompts using production trace data, evaluations, and annotations... runs a data-driven optimization loop using the ax CLI") and the 'when' is explicit with concrete trigger phrases ("Use when the user mentions optimize prompt, improve prompt, ..."). This mirrors the level-5 anchor example almost exactly; the level-4 anchor requires the 'when' to be less explicit, which is not the case here.

5 / 5

Trigger Term Quality

The trigger clause includes seven natural phrases users would say: "optimize prompt, improve prompt, make AI respond better, improve output quality, prompt engineering, prompt tuning, or system prompt improvement" — covering synonyms and colloquial variations ("make AI respond better" is an especially natural phrasing). This matches the comprehensive-synonyms anchor; file extensions are not applicable to this domain. It is above the level-4 anchor, which expects natural terms to be missing.

5 / 5

Distinctiveness Conflict Risk

The niche is clear (data-driven prompt optimization via Arize trace data and the ax CLI), which distinguishes it from generic skills, but a few trigger phrases — "make AI respond better" and "improve output quality" — are broad enough to overlap with general LLM-output-quality or prompt-writing skills. This fits 'Mostly distinct; minor overlap risk with closely related skills'; it does not reach level 5 because those generic triggers lack the Arize/prompt-optimization specificity that keeps conflict risk minimal.

4 / 5

Total

19

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

github/awesome-copilot

Path

skills/arize-prompt-optimization/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.