metric-pack-designer

Design custom metric packs for plugin-eval so teams can add local evaluation rubrics that emit schema-compatible checks and metrics. Use when the user wants their own evaluation criteria or visualizations.

Quality

75%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./plugins/plugin-eval/skills/metric-pack-designer/SKILL.md

Metric Pack Designer

Use this skill when the user wants to extend plugin-eval with a local rubric.

Workflow

Clarify the custom rubric categories and target kinds.
Define the smallest useful checks[] and metrics[] payload.
Create a metric-pack manifest plus a script that prints JSON to stdout.
Run the pack through plugin-eval analyze <path> --metric-pack <manifest.json>.

Design Rules

Keep IDs stable across runs so comparisons stay meaningful.
Emit only checks[], metrics[], and optional artifacts[].
Do not try to overwrite the core score or summary.
Prefer deterministic local signals over subjective text generation.

Reference

../../references/metric-pack-manifest.md

Repository: openai/plugins
Path: plugins/plugin-eval/skills/metric-pack-designer/SKILL.md
Commit: 11c74d6

Last updated: 17 days ago
First committed: 4 months ago

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.