tessl install tessl/pypi-crosshair-tool@0.0.0Analyze Python code for correctness using symbolic execution and SMT solving to automatically find counterexamples for functions with type annotations and contracts.
Agent Success
Agent success rate when using this tile
86%
Improvement
Agent success rate improvement when using this tile compared to baseline
1.25x
Baseline
Agent success rate without this tile
69%
{
"context": "This evaluation assesses how well the engineer uses crosshair-tool's plugin system to register custom types and contracts. The focus is on using register_type() for symbolic value creation and register_contract() for programmatic contract registration.",
"type": "weighted_checklist",
"checklist": [
{
"name": "register_type usage",
"description": "Uses crosshair's register_type() function to register a symbolic value creator for the Coordinate type",
"max_score": 25
},
{
"name": "Symbolic value creator",
"description": "Implements a creator function that returns symbolic Coordinate instances using proxy_for_type() or similar crosshair APIs for creating symbolic floats",
"max_score": 20
},
{
"name": "Constraint application",
"description": "Applies proper constraints to symbolic latitude ([-90, 90]) and longitude ([-180, 180]) values using Z3 or crosshair's constraint mechanisms",
"max_score": 15
},
{
"name": "register_contract usage",
"description": "Uses crosshair's register_contract() function to programmatically register contracts for the calculate_distance function",
"max_score": 25
},
{
"name": "Precondition specification",
"description": "Specifies a precondition function that validates both arguments are Coordinate instances (checking type or attributes)",
"max_score": 8
},
{
"name": "Postcondition specification",
"description": "Specifies a postcondition function that validates the return value is non-negative (>= 0)",
"max_score": 7
}
]
}