or run

tessl search
Log in

Version

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/safety@3.6.x
tile.json

tessl/pypi-safety

tessl install tessl/pypi-safety@3.6.0

Scan dependencies for known vulnerabilities and licenses.

Agent Success

Agent success rate when using this tile

61%

Improvement

Agent success rate improvement when using this tile compared to baseline

1.39x

Baseline

Agent success rate without this tile

44%

rubric.jsonevals/scenario-6/

{
  "context": "This criteria evaluates how well the engineer uses the Safety CLI package's GitHub integration capabilities to scan for vulnerabilities and create automated pull requests for remediation.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Vulnerability Scanning",
      "description": "Uses Safety's scan functionality (e.g., safety.scan.command.scan() or safety.safety.check()) to detect vulnerabilities in the requirements file.",
      "max_score": 25
    },
    {
      "name": "GitHub PR Creation",
      "description": "Uses Safety's GitHub integration functions (e.g., safety.alerts.github.github_pr()) to create pull requests for vulnerable packages.",
      "max_score": 30
    },
    {
      "name": "Remediation Application",
      "description": "Uses Safety's remediation capabilities (e.g., apply_fixes() or calculate_remediations()) to determine updated package versions for fixing vulnerabilities.",
      "max_score": 20
    },
    {
      "name": "Severity Filtering",
      "description": "Implements CVSS severity-based filtering using Safety's severity scoring (ignore-cvss-severity-below or similar) to only create PRs for vulnerabilities meeting the threshold.",
      "max_score": 15
    },
    {
      "name": "GitHub Authentication",
      "description": "Properly configures GitHub authentication using the provided token for API access with Safety's GitHub integration.",
      "max_score": 10
    }
  ]
}