tessl install tessl/pypi-safety@3.6.0Scan dependencies for known vulnerabilities and licenses.
Agent Success
Agent success rate when using this tile
61%
Improvement
Agent success rate improvement when using this tile compared to baseline
1.39x
Baseline
Agent success rate without this tile
44%
{
"context": "This criteria evaluates how well the engineer uses the Safety CLI package to capture and process security announcements and warnings. The focus is on proper invocation of the scanner, extraction of announcements and notifications, and correct handling of different warning types.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Safety CLI Invocation",
"description": "Uses the safety scan command or safety check command to execute vulnerability scanning. Should invoke safety as a subprocess or use the safety API directly.",
"max_score": 25
},
{
"name": "JSON Output Format",
"description": "Utilizes safety's --output json flag or --output screen flag to get structured output that contains announcements and warnings. Properly configures output format for parsing.",
"max_score": 15
},
{
"name": "Announcement Extraction",
"description": "Correctly extracts security announcements from safety's output. Safety provides announcements through get_announcements() function or in scan output. Should capture platform notifications and security announcements.",
"max_score": 20
},
{
"name": "Warning Categorization",
"description": "Properly identifies and categorizes local warnings from safety's output including unpinned requirements warnings, invalid version warnings, and other scan-related warnings via add_local_notifications() or output parsing.",
"max_score": 20
},
{
"name": "Output Parsing",
"description": "Correctly parses safety's output format to extract relevant data. Should handle different output modes (JSON, screen) and parse announcements and warnings from the structured output.",
"max_score": 15
},
{
"name": "Error Handling",
"description": "Properly handles safety command execution failures, missing dependencies, or invalid target paths. Uses appropriate exception handling for safety CLI errors.",
"max_score": 5
}
]
}