or run

tessl search
Log in

Version

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/safety@3.6.x
tile.json

tessl/pypi-safety

tessl install tessl/pypi-safety@3.6.0

Scan dependencies for known vulnerabilities and licenses.

Agent Success

Agent success rate when using this tile

61%

Improvement

Agent success rate improvement when using this tile compared to baseline

1.39x

Baseline

Agent success rate without this tile

44%

rubric.jsonevals/scenario-1/

{
  "context": "This criteria evaluates how well an engineer uses the Safety CLI package's command intention normalization capabilities to parse and normalize package manager commands. The focus is on using Safety's abstraction layer (CommandToolIntention, Dependency classes, and ToolIntentionType enum) to handle commands across different package managers.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Import Safety components",
      "description": "Uses Safety CLI's command intention normalization components: imports from safety.tool.intents (CommandToolIntention, ToolIntentionType) and safety.tool.base (Dependency) or equivalent modules for command parsing.",
      "max_score": 25
    },
    {
      "name": "CommandToolIntention usage",
      "description": "Correctly instantiates and uses CommandToolIntention class to represent parsed commands with appropriate ToolIntentionType enum values (ADD_PACKAGE, REMOVE_PACKAGE, UPDATE_PACKAGE).",
      "max_score": 25
    },
    {
      "name": "Dependency extraction",
      "description": "Properly uses the Dependency class to represent package information including name, version constraints, and extras extracted from commands.",
      "max_score": 20
    },
    {
      "name": "Multi-tool support",
      "description": "Leverages Safety's tool resolver or similar functionality to correctly parse commands from different package managers (pip, poetry, npm) into normalized intentions.",
      "max_score": 20
    },
    {
      "name": "Dev dependency detection",
      "description": "Correctly identifies and flags development dependencies (e.g., poetry's --group dev, npm's --save-dev) using Safety's command parsing capabilities.",
      "max_score": 10
    }
  ]
}