or run

tessl search
Log in

Version

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/safety@3.6.x
tile.json

tessl/pypi-safety

tessl install tessl/pypi-safety@3.6.0

Scan dependencies for known vulnerabilities and licenses.

Agent Success

Agent success rate when using this tile

61%

Improvement

Agent success rate improvement when using this tile compared to baseline

1.39x

Baseline

Agent success rate without this tile

44%

rubric.jsonevals/scenario-7/

{
  "context": "This criteria evaluates how well the engineer uses the Safety CLI package to implement system-wide Python environment vulnerability scanning. The focus is on proper usage of Safety's scanning commands and output handling capabilities.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "System-wide scanning",
      "description": "Uses 'safety system-scan' or 'safety scan' command to perform system-wide vulnerability scanning when no target paths are provided",
      "max_score": 30
    },
    {
      "name": "Targeted directory scanning",
      "description": "Uses 'safety scan --target' or 'safety system-scan --target' with specified directory paths to scan specific locations",
      "max_score": 25
    },
    {
      "name": "Output format handling",
      "description": "Uses '--output json' flag or equivalent to get structured output from Safety CLI that can be parsed programmatically",
      "max_score": 20
    },
    {
      "name": "Subprocess execution",
      "description": "Uses subprocess module (subprocess.run, subprocess.check_output, or similar) to execute Safety CLI commands and capture output",
      "max_score": 15
    },
    {
      "name": "Report data extraction",
      "description": "Parses Safety's JSON output to extract vulnerability information including package counts and vulnerability details",
      "max_score": 10
    }
  ]
}