CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-crosshair-tool

Analyze Python code for correctness using symbolic execution and SMT solving to automatically find counterexamples for functions with type annotations and contracts.

86

1.24x
Overview
Eval results
Files

rubric.jsonevals/scenario-7/

{
  "context": "This evaluation assesses how well the engineer uses crosshair-tool's plugin system to register custom types and contracts. The focus is on using register_type() for symbolic value creation and register_contract() for programmatic contract registration.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "register_type usage",
      "description": "Uses crosshair's register_type() function to register a symbolic value creator for the Coordinate type",
      "max_score": 25
    },
    {
      "name": "Symbolic value creator",
      "description": "Implements a creator function that returns symbolic Coordinate instances using proxy_for_type() or similar crosshair APIs for creating symbolic floats",
      "max_score": 20
    },
    {
      "name": "Constraint application",
      "description": "Applies proper constraints to symbolic latitude ([-90, 90]) and longitude ([-180, 180]) values using Z3 or crosshair's constraint mechanisms",
      "max_score": 15
    },
    {
      "name": "register_contract usage",
      "description": "Uses crosshair's register_contract() function to programmatically register contracts for the calculate_distance function",
      "max_score": 25
    },
    {
      "name": "Precondition specification",
      "description": "Specifies a precondition function that validates both arguments are Coordinate instances (checking type or attributes)",
      "max_score": 8
    },
    {
      "name": "Postcondition specification",
      "description": "Specifies a postcondition function that validates the return value is non-negative (>= 0)",
      "max_score": 7
    }
  ]
}

Install with Tessl CLI

npx tessl i tessl/pypi-crosshair-tool

tile.json