Scan dependencies for known vulnerabilities and licenses.
61
Pending
Does it follow best practices?
Impact
61%
1.38xAverage score across 10 eval scenarios
Pending
The risk profile of this skill
{
"context": "This criteria evaluates how well the engineer uses the Safety CLI package to implement system-wide Python environment vulnerability scanning. The focus is on proper usage of Safety's scanning commands and output handling capabilities.",
"type": "weighted_checklist",
"checklist": [
{
"name": "System-wide scanning",
"description": "Uses 'safety system-scan' or 'safety scan' command to perform system-wide vulnerability scanning when no target paths are provided",
"max_score": 30
},
{
"name": "Targeted directory scanning",
"description": "Uses 'safety scan --target' or 'safety system-scan --target' with specified directory paths to scan specific locations",
"max_score": 25
},
{
"name": "Output format handling",
"description": "Uses '--output json' flag or equivalent to get structured output from Safety CLI that can be parsed programmatically",
"max_score": 20
},
{
"name": "Subprocess execution",
"description": "Uses subprocess module (subprocess.run, subprocess.check_output, or similar) to execute Safety CLI commands and capture output",
"max_score": 15
},
{
"name": "Report data extraction",
"description": "Parses Safety's JSON output to extract vulnerability information including package counts and vulnerability details",
"max_score": 10
}
]
}docs
evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10