meterian/security-audit
Use for dependency security audits and compliance checks. Use when auditing project dependencies for vulnerabilities, answering "is [library] [version] safe?" questions, or remediating vulnerable libraries. Also activates automatically when the user opens or modifies a manifest file (package.json, package-lock.json, yarn.lock, pnpm-lock.yaml, requirements.txt, pom.xml, Cargo.toml, go.mod, Gemfile, composer.json, build.gradle, *.csproj, pubspec.yaml, conanfile.txt, conanfile.py, project.clj, deps.edn, Package.swift, pubspec.lock, Package.resolved, Gemfile.lock, poetry.lock, uv.lock, Cargo.lock, composer.lock).
96
1.83x
Quality
90%
Does it follow best practices?
Impact
99%
1.83xAverage score across 8 eval scenarios
Securityby
Passed
No known issues
criteria.jsonevals/scenario-8/
{
"context": "Tests whether the agent discovers all three manifest files, assigns the correct language to each ecosystem, and combines all packages into a single Meterian CLI check invocation. Verified via scan-raw.json containing all three language types in one file.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Finds all manifests",
"description": "Scans for all three manifest types: package.json (Node.js), Cargo.toml (Rust), and requirements.txt (Python)",
"max_score": 12
},
{
"name": "nodejs language",
"description": "`scan-raw.json` contains entries with `\"language\":\"nodejs\"` for packages from package.json",
"max_score": 12
},
{
"name": "rust language",
"description": "`scan-raw.json` contains entries with `\"language\":\"rust\"` for packages from Cargo.toml (not `cargo` or `rs`)",
"max_score": 12
},
{
"name": "python language",
"description": "`scan-raw.json` contains entries with `\"language\":\"python\"` for packages from requirements.txt (not `pip` or `py`)",
"max_score": 12
},
{
"name": "Raw scan data file",
"description": "A file named `scan-raw.json` exists containing a JSON object with a `vulnerable` array covering all three ecosystems in a single scan",
"max_score": 15
},
{
"name": "Single scan",
"description": "A single `scan-raw.json` covers packages from all three ecosystems — not separate files per language — confirming one combined CLI invocation",
"max_score": 12
},
{
"name": "Markdown table",
"description": "Report contains a markdown table with the required audit columns (Package, Version, Severity, ID, Safe Version(s))",
"max_score": 13
},
{
"name": "Summary line",
"description": "Report includes a summary line referencing total vulnerability count and package counts",
"max_score": 12
}
]
}