or run

tessl search
Log in

Version

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/safety@3.6.x
tile.json

tessl/pypi-safety

tessl install tessl/pypi-safety@3.6.0

Scan dependencies for known vulnerabilities and licenses.

Agent Success

Agent success rate when using this tile

61%

Improvement

Agent success rate improvement when using this tile compared to baseline

1.39x

Baseline

Agent success rate without this tile

44%

rubric.jsonevals/scenario-4/

{
  "context": "This criteria evaluates how well the engineer uses the Safety CLI package to capture and process security announcements and warnings. The focus is on proper invocation of the scanner, extraction of announcements and notifications, and correct handling of different warning types.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Safety CLI Invocation",
      "description": "Uses the safety scan command or safety check command to execute vulnerability scanning. Should invoke safety as a subprocess or use the safety API directly.",
      "max_score": 25
    },
    {
      "name": "JSON Output Format",
      "description": "Utilizes safety's --output json flag or --output screen flag to get structured output that contains announcements and warnings. Properly configures output format for parsing.",
      "max_score": 15
    },
    {
      "name": "Announcement Extraction",
      "description": "Correctly extracts security announcements from safety's output. Safety provides announcements through get_announcements() function or in scan output. Should capture platform notifications and security announcements.",
      "max_score": 20
    },
    {
      "name": "Warning Categorization",
      "description": "Properly identifies and categorizes local warnings from safety's output including unpinned requirements warnings, invalid version warnings, and other scan-related warnings via add_local_notifications() or output parsing.",
      "max_score": 20
    },
    {
      "name": "Output Parsing",
      "description": "Correctly parses safety's output format to extract relevant data. Should handle different output modes (JSON, screen) and parse announcements and warnings from the structured output.",
      "max_score": 15
    },
    {
      "name": "Error Handling",
      "description": "Properly handles safety command execution failures, missing dependencies, or invalid target paths. Uses appropriate exception handling for safety CLI errors.",
      "max_score": 5
    }
  ]
}