Scan dependencies for known vulnerabilities and licenses.
Overall
score
61%
{
"context": "This criteria evaluates how well the engineer uses the Safety CLI package's GitHub integration capabilities to scan for vulnerabilities and create automated pull requests for remediation.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Vulnerability Scanning",
"description": "Uses Safety's scan functionality (e.g., safety.scan.command.scan() or safety.safety.check()) to detect vulnerabilities in the requirements file.",
"max_score": 25
},
{
"name": "GitHub PR Creation",
"description": "Uses Safety's GitHub integration functions (e.g., safety.alerts.github.github_pr()) to create pull requests for vulnerable packages.",
"max_score": 30
},
{
"name": "Remediation Application",
"description": "Uses Safety's remediation capabilities (e.g., apply_fixes() or calculate_remediations()) to determine updated package versions for fixing vulnerabilities.",
"max_score": 20
},
{
"name": "Severity Filtering",
"description": "Implements CVSS severity-based filtering using Safety's severity scoring (ignore-cvss-severity-below or similar) to only create PRs for vulnerabilities meeting the threshold.",
"max_score": 15
},
{
"name": "GitHub Authentication",
"description": "Properly configures GitHub authentication using the provided token for API access with Safety's GitHub integration.",
"max_score": 10
}
]
}Install with Tessl CLI
npx tessl i tessl/pypi-safetydocs
evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10