CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-crosshair-tool

Analyze Python code for correctness using symbolic execution and SMT solving to automatically find counterexamples for functions with type annotations and contracts.

86

1.24x
Overview
Eval results
Files

rubric.jsonevals/scenario-3/

{
  "context": "This criteria evaluates how well the engineer uses CrossHair's contract verification features to enforce business rules in a bank account system. The focus is on proper use of preconditions, postconditions, and class invariants using any supported contract syntax.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Preconditions for inputs",
      "description": "Uses preconditions to validate input parameters (positive amounts, non-negative initial balance, sufficient funds). Should use contract syntax like `pre:` in docstrings, `@icontract.require`, `@deal.pre`, or leading assertions.",
      "max_score": 25
    },
    {
      "name": "Postconditions for state",
      "description": "Uses postconditions to verify state changes after operations (balance increases after deposit, decreases after withdrawal, transfer updates both accounts). Should use `post:` in docstrings, `@icontract.ensure`, `@deal.post`, or similar contract mechanisms.",
      "max_score": 25
    },
    {
      "name": "Class invariants",
      "description": "Defines class invariants to ensure balance remains non-negative throughout the object's lifetime. Should use `inv:` in docstrings, `@icontract.invariant`, or equivalent mechanisms.",
      "max_score": 20
    },
    {
      "name": "Old value tracking",
      "description": "Uses `__old__` variable or equivalent mechanisms in postconditions to reference original argument values when verifying state changes (e.g., verifying balance increased by deposit amount).",
      "max_score": 15
    },
    {
      "name": "Contract syntax variety",
      "description": "Demonstrates understanding of multiple contract syntax options by using appropriate contract specifications (PEP316 docstrings, icontract decorators, deal decorators, or assertions) consistently throughout the implementation.",
      "max_score": 15
    }
  ]
}

Install with Tessl CLI

npx tessl i tessl/pypi-crosshair-tool

tile.json