testland/ragas-evaluation
Authors and runs Ragas - RAG-pipeline evaluation framework with metrics organized into RAG (Faithfulness, Response Relevancy, Context Precision/Recall, Context Entities Recall, Noise Sensitivity), Natural Language Comparison (Factual Correctness, Semantic Similarity, BLEU/ROUGE/CHRF/Exact Match), Agents/Tool-Use (Topic Adherence, Tool Call Accuracy/F1, Agent Goal Accuracy), General Purpose (Aspect Critic, Rubrics-based Scoring), Nvidia (Answer Accuracy, Context Relevance, Response Groundedness), and Summarization. Use when the user evaluates a RAG pipeline (retriever + generator) and needs the deepest metric variety in the OSS LLM-eval space.
80
100%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
View guide
Low
Low-risk findings worth noting
Low
Low-risk findings.
2 low severity findings. Worth noting, but not necessarily harmful.
W011: Third-party content exposure detected (indirect prompt injection risk).
What this means
The skill exposes the agent to untrusted, user-generated content from public third-party sources, creating a risk of indirect prompt injection. This includes browsing arbitrary URLs, reading social media posts or forum comments, and analyzing content from unknown websites.
Why it was flagged
The required runtime workflow ingests user/outside-authored evaluation dataset text (e.g., `question`, `answer`, and especially retrieved `contexts`) and then LLM-judges it via `evaluate()`/metrics like Faithfulness and rubric-based scoring.
W012: Unverifiable external dependency detected (runtime URL that controls agent).
What this means
The skill fetches instructions or code from an external URL at runtime, and the fetched content directly controls the agent’s prompts or executes code. This dynamic dependency allows the external source to modify the agent’s behavior without any changes to the skill itself.
Why it was flagged
The skill explicitly instructs installing remote code via "pip install git+https://github.com/explodinggradients/ragas", which fetches and installs/executes third‑party code that the evaluation runtime will run and that can control prompts/metrics.
- Audited
- Security analysis