CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-syrupy

Pytest snapshot testing utility that enables developers to write tests asserting immutability of computed results.

Overall
score

80%

Overview
Eval results
Files

rubric.jsonevals/scenario-1/

{
  "context": "Evaluates use of syrupy snapshot testing to enforce type-stable run summaries and handle corrupted snapshot files through regeneration. Focuses on the pytest snapshot fixture, strict matchers for nested metrics, and tainted snapshot recovery with the update flag.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Snapshot fixture",
      "description": "Summary comparisons use the syrupy `snapshot` pytest fixture rather than manual serialization, including multi-assert handling for the baseline summary.",
      "max_score": 20
    },
    {
      "name": "Strict matcher",
      "description": "Uses `path_type` (or equivalent syrupy matcher) with `strict=True` to enforce `metrics.duration_ms` as an int and configures it on the `snapshot` assertion.",
      "max_score": 25
    },
    {
      "name": "Type error check",
      "description": "Negative test expects a `StrictPathTypeError`/`PathTypeError` when `duration_ms` is a string and asserts the error references the `metrics.duration_ms` path.",
      "max_score": 15
    },
    {
      "name": "Tainted detection",
      "description": "Intentionally tampering with the stored snapshot leads to a `TaintedSnapshotError` from syrupy, and the test asserts the message directs the user to regenerate snapshots.",
      "max_score": 20
    },
    {
      "name": "Regeneration update",
      "description": "Running with `--snapshot-update` (or syrupy `with_defaults(update=True)`) rewrites the tainted snapshot and subsequent `snapshot` assertions pass without manual cleanup.",
      "max_score": 20
    }
  ]
}

Install with Tessl CLI

npx tessl i tessl/pypi-syrupy

tile.json