CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-safety

Scan dependencies for known vulnerabilities and licenses.

Overall
score

61%

Overview
Eval results
Files

rubric.jsonevals/scenario-1/

{
  "context": "This criteria evaluates how well an engineer uses the Safety CLI package's command intention normalization capabilities to parse and normalize package manager commands. The focus is on using Safety's abstraction layer (CommandToolIntention, Dependency classes, and ToolIntentionType enum) to handle commands across different package managers.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Import Safety components",
      "description": "Uses Safety CLI's command intention normalization components: imports from safety.tool.intents (CommandToolIntention, ToolIntentionType) and safety.tool.base (Dependency) or equivalent modules for command parsing.",
      "max_score": 25
    },
    {
      "name": "CommandToolIntention usage",
      "description": "Correctly instantiates and uses CommandToolIntention class to represent parsed commands with appropriate ToolIntentionType enum values (ADD_PACKAGE, REMOVE_PACKAGE, UPDATE_PACKAGE).",
      "max_score": 25
    },
    {
      "name": "Dependency extraction",
      "description": "Properly uses the Dependency class to represent package information including name, version constraints, and extras extracted from commands.",
      "max_score": 20
    },
    {
      "name": "Multi-tool support",
      "description": "Leverages Safety's tool resolver or similar functionality to correctly parse commands from different package managers (pip, poetry, npm) into normalized intentions.",
      "max_score": 20
    },
    {
      "name": "Dev dependency detection",
      "description": "Correctly identifies and flags development dependencies (e.g., poetry's --group dev, npm's --save-dev) using Safety's command parsing capabilities.",
      "max_score": 10
    }
  ]
}

Install with Tessl CLI

npx tessl i tessl/pypi-safety

tile.json