CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-crosshair-tool

Analyze Python code for correctness using symbolic execution and SMT solving to automatically find counterexamples for functions with type annotations and contracts.

86

1.24x
Overview
Eval results
Files

rubric.jsonevals/scenario-1/

{
  "context": "This evaluation assesses the engineer's ability to use CrossHair's symbolic execution capabilities to verify generic type constraints, particularly focusing on TypeVar binding, bounded TypeVars, and contract verification using PEP316-style contracts with preconditions and postconditions.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "TypeVar declaration",
      "description": "Correctly declares TypeVar 'T' and bounded TypeVar 'NumericT' with appropriate constraints (int, float) at module level",
      "max_score": 10
    },
    {
      "name": "Generic tuple typing",
      "description": "Uses Tuple[T, T] type annotation in swap_pair to enforce same-type constraint for both tuple elements",
      "max_score": 15
    },
    {
      "name": "PEP316 preconditions",
      "description": "Implements preconditions using 'pre:' in docstrings for all four functions (swap_pair, process_numeric, get_first, transform_list) to validate inputs",
      "max_score": 15
    },
    {
      "name": "PEP316 postconditions",
      "description": "Implements postconditions using 'post:' in docstrings with __return__ references to verify output correctness and type preservation",
      "max_score": 20
    },
    {
      "name": "Bounded TypeVar usage",
      "description": "Uses NumericT bounded TypeVar correctly in process_numeric function signature for both parameters and return type to restrict to numeric types",
      "max_score": 15
    },
    {
      "name": "Generic List typing",
      "description": "Uses List[T] type annotations in get_first and transform_list to maintain generic type relationships between input lists and return values",
      "max_score": 15
    },
    {
      "name": "Contract verification",
      "description": "Contracts are verifiable by CrossHair's symbolic execution (can be checked with 'crosshair check' command without errors in contract syntax)",
      "max_score": 10
    }
  ]
}

Install with Tessl CLI

npx tessl i tessl/pypi-crosshair-tool

tile.json