CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-crosshair-tool

Analyze Python code for correctness using symbolic execution and SMT solving to automatically find counterexamples for functions with type annotations and contracts.

86

1.24x
Overview
Eval results
Files

rubric.jsonevals/scenario-2/

{
  "context": "This criteria evaluates how well the engineer uses CrossHair's symbolic type system to generate symbolic values for different Python types. The focus is on correct usage of proxy_for_type, realize/deep_realize, and understanding of symbolic value creation.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Uses proxy_for_type",
      "description": "The implementation imports and uses proxy_for_type() from crosshair.libimpl.builtinslib or crosshair.core_and_libs to create symbolic values",
      "max_score": 30
    },
    {
      "name": "Correct type handling",
      "description": "The create_symbolic() method correctly passes the type_hint and name parameters to proxy_for_type(), handling both primitive types (int, str, bool) and collection types (list, dict)",
      "max_score": 25
    },
    {
      "name": "Uses realize function",
      "description": "The realize() method imports and uses realize() or deep_realize() from crosshair.libimpl.builtinslib or crosshair.core_and_libs to convert symbolic values to concrete values",
      "max_score": 20
    },
    {
      "name": "Proper error handling",
      "description": "The implementation handles TypeError or other exceptions appropriately when unsupported types are passed to create_symbolic(), as specified in the API documentation",
      "max_score": 15
    },
    {
      "name": "Symbolic value integration",
      "description": "The implementation correctly returns symbolic values that maintain symbolic properties and can be used in constraint solving contexts, not just regular Python values",
      "max_score": 10
    }
  ]
}

Install with Tessl CLI

npx tessl i tessl/pypi-crosshair-tool

tile.json