CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-safety

Scan dependencies for known vulnerabilities and licenses.

Overall
score

61%

Overview
Eval results
Files

rubric.jsonevals/scenario-6/

{
  "context": "This criteria evaluates how well the engineer uses the Safety CLI package's GitHub integration capabilities to scan for vulnerabilities and create automated pull requests for remediation.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Vulnerability Scanning",
      "description": "Uses Safety's scan functionality (e.g., safety.scan.command.scan() or safety.safety.check()) to detect vulnerabilities in the requirements file.",
      "max_score": 25
    },
    {
      "name": "GitHub PR Creation",
      "description": "Uses Safety's GitHub integration functions (e.g., safety.alerts.github.github_pr()) to create pull requests for vulnerable packages.",
      "max_score": 30
    },
    {
      "name": "Remediation Application",
      "description": "Uses Safety's remediation capabilities (e.g., apply_fixes() or calculate_remediations()) to determine updated package versions for fixing vulnerabilities.",
      "max_score": 20
    },
    {
      "name": "Severity Filtering",
      "description": "Implements CVSS severity-based filtering using Safety's severity scoring (ignore-cvss-severity-below or similar) to only create PRs for vulnerabilities meeting the threshold.",
      "max_score": 15
    },
    {
      "name": "GitHub Authentication",
      "description": "Properly configures GitHub authentication using the provided token for API access with Safety's GitHub integration.",
      "max_score": 10
    }
  ]
}

Install with Tessl CLI

npx tessl i tessl/pypi-safety

tile.json