giskard-llm@1.3.6 • testland • Registry • Tessl

PluginPublic

testland/giskard-llm

Use when an LLM app needs adversarial or red-team tests on top of functional evals. Runs giskard.scan on any model wrapped as giskard.Model to probe hallucination, harmful content, prompt injection, sensitive data disclosure, stereotypes, robustness and sycophancy, with an HTML report.

Quality

85%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

75%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable body with a clear six-step workflow, executable code throughout, and disciplined token usage. It consistently sits just below excellent: a dated inline version note, a severity/level naming mismatch in the CI gating snippet, no explicit post-scan validation checkpoint, and no use of bundle files for the bulkier catalog material.

Suggestions

Reconcile the CI gating snippet with its prose: the text says to parse findings for "severity" but the code filters on issue.level == "major" — use the actual scan_results API field consistently so the snippet is copy-paste correct.

Move the dated version note (2026-05-06, v2-vs-v3 maintenance status) into a clearly labeled 'Versioning / deprecated' section so time-sensitive details don't penalize the main flow, and trim the repeated 'Per [gk-gh]' attributions to one citation per section.

Add an explicit validation checkpoint after Step 3 (e.g., confirm the HTML report was written and scan_results contains issues before exporting/generating the test suite), and consider splitting the Step 4 detector catalog and Step 6 CI details into a one-level-deep reference file.

Dimension	Reasoning	Score
Conciseness	The body is dense, code-first, and assumes Claude's competence (no library tutorials or concept explanations), but the dated inline "Important version note (2026-05-06)" is time-sensitive content outside a deprecated/old-patterns section and the repeated "Per [gk-gh][gk-gh]" attributions pad slightly — minor trims, matching anchor 4 rather than the every-token-earns-its-place anchor 5.	4 / 5
Actionability	Nearly everything is executable: a pinned install command ("giskard[llm]>2,<3"), a complete giskard.Model wrapping block, scan/HTML-export/test-suite code, and CI YAML. Minor gaps keep it from 5: the CI gating snippet tests "issue.level == \"major\"" while the prose says to parse "severity" (a mismatch the reader must reconcile), and my_llm_chain is a placeholder.	4 / 5
Workflow Clarity	Steps 1-6 (Install, Wrap, Scan, Detector catalog, Test suites, CI) form a clear, well-ordered sequence with an explicit CI severity-gating checkpoint (sys.exit(1) on major findings). It falls short of anchor 5 only because there is no explicit validation step verifying the scan completed and the report was generated before moving to CI integration.	4 / 5
Progressive Disclosure	No bundle files exist, and the ~180-line body is cleanly sectioned (Overview, When to use, Steps 1-6, Anti-patterns, Limitations, References) with everything appropriately inline at this size and external links clearly listed. It does not qualify for the under-50-line simple-skill exception to reach 5, and the detector catalog plus CI details could live in a one-level-deep reference file.	4 / 5
	Total	16 / 20 Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description that explicitly states both what the skill does and when to use it, with tool-specific, concrete capability coverage. Trigger-term and distinctiveness are good rather than excellent due to missing synonyms (vulnerability scanning, safety) and minor overlap with functional-eval skills on detector terms.

Dimension	Reasoning	Score
Specificity	"Runs giskard.scan" on models "wrapped as giskard.Model" to "probe hallucination, harmful content, prompt injection, sensitive data disclosure, stereotypes, robustness and sycophancy, with an HTML report" — multiple concrete actions with comprehensive, tool-specific coverage in third-person voice; no gaps that would drop it to 4.	5 / 5
Completeness	Explicitly answers 'when' ("Use when an LLM app needs adversarial or red-team tests on top of functional evals") and 'what' (runs giskard.scan, probes seven named vulnerability categories, produces an HTML report) with concrete trigger phrases; the 'when' clause is explicit and specific, not merely present.	5 / 5
Trigger Term Quality	Good natural keyword coverage ("adversarial or red-team tests", "hallucination", "prompt injection", "LLM app") that users would actually say, but misses common synonyms such as "vulnerability scanning", "safety", or "security testing" — not the comprehensive synonym coverage of anchor 5.	4 / 5
Distinctiveness Conflict Risk	The framing "adversarial or red-team tests on top of functional evals" plus tool-specific names (giskard.scan, giskard.Model) carves a mostly distinct niche, but detector terms like "hallucination" and "harmful content" create minor overlap risk with functional-eval skills (DeepEval, Ragas), matching anchor 4 rather than 5.	4 / 5
	Total	18 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

5 days ago

testland/giskard-llm

On this page