Scan dependencies for known vulnerabilities and licenses.
Overall
score
61%
{
"context": "This evaluation assesses how effectively the engineer utilizes the Safety CLI package's output formatting capabilities to implement a vulnerability report formatter. The focus is on proper use of Safety's formatter classes and methods, not on general code quality or structure.",
"type": "weighted_checklist",
"checklist": [
{
"name": "SafetyFormatter usage",
"description": "Uses Safety's SafetyFormatter base class or inherits from formatter classes in safety.formatter or safety.formatters modules",
"max_score": 25
},
{
"name": "JSON formatter implementation",
"description": "Utilizes JsonReport class from safety.formatters.json or similar JSON formatting functionality from Safety package for JSON output",
"max_score": 20
},
{
"name": "Screen formatter implementation",
"description": "Uses ScreenReport class from safety.formatters.screen or Rich terminal formatting capabilities provided by Safety for terminal output",
"max_score": 20
},
{
"name": "Text formatter implementation",
"description": "Leverages TextReport or BareReport classes from safety.formatter for plain text output generation",
"max_score": 15
},
{
"name": "Output format selection",
"description": "Correctly maps format options (json/screen/text) to appropriate Safety formatter classes or output format constants",
"max_score": 10
},
{
"name": "Vulnerability data models",
"description": "Uses Safety's data models (e.g., from safety.models or safety.scan.models) to represent vulnerability data rather than plain dictionaries",
"max_score": 10
}
]
}Install with Tessl CLI
npx tessl i tessl/pypi-safetydocs
evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10