llm-as-judge-evaluation
Evaluate LLM outputs using frontier models as judges. Use for pairwise model comparison, quality scoring with custom rubrics, and automated evaluation pipelines. Covers position bias mitigation, statistical significance, and generating preference data for DPO/RLHF.
69
87%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
Low
Low-risk findings worth noting
W011: Third-party content exposure detected (indirect prompt injection risk).
What this means
The skill exposes the agent to untrusted, user-generated content from public third-party sources, creating a risk of indirect prompt injection. This includes browsing arbitrary URLs, reading social media posts or forum comments, and analyzing content from unknown websites.
Why it was flagged
The LLM runtime prompt includes {user_input}, {response_a}, and {response_b}—which are free-form text supplied to the evaluation functions by the caller—so if those strings originate from outsiders (e.g., eval_set items or model outputs containing outsider-authored content), that text is fed verbatim into the judge LLM context via `client.chat.completions.create(..., messages=[{"content": prompt}])`.
- Audited
- Security analysis
Is this your skill?
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.