CtrlK
CommunityDocumentationLog inGet started
Tessl Logo

tessl/pypi-vedro

Pragmatic Testing Framework for Python with BDD-style syntax and pluggable architecture

49%

Overall

Evaluation49%

1.09x

Agent success when using this tile

Overview
Eval results
Files

rubric.jsonevals/scenario-3/

{
  "context": "Evaluates whether the checkout scenarios are expressed with Vedro's scenario DSL: a class-based flow with subject and step methods plus two decorator-driven flows built from given/when/then functions. It prioritizes correct use of Scenario subclassing, @scenario registration, and step decorators over general code style.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Class scenario",
      "description": "CheckoutSuccessFlow subclasses vedro.Scenario (or uses Scenario re-export), defines a subject string, and registers its given/when/then steps with @given/@when/@then decorators in the class-based style.",
      "max_score": 25
    },
    {
      "name": "Decorator scenario",
      "description": "payment_retry_flow is declared with @scenario and composes @given, @when, and @then functions that the runner can discover without relying on a class.",
      "max_score": 20
    },
    {
      "name": "Retry logging",
      "description": "The retry flow uses Vedro step ordering (given → when → then) to log the first failed charge and the succeeding retry, keeping attempt details in shared state accessible across steps.",
      "max_score": 15
    },
    {
      "name": "Out-of-stock flow",
      "description": "out_of_stock_flow is registered via @scenario with @given/@when/@then; the when-step blocks payment work when inventory is short and the then-step surfaces the failure reason, using Vedro assertions to fail the scenario.",
      "max_score": 15
    },
    {
      "name": "State sharing",
      "description": "Inventory, orders, totals, and events are stored on the Scenario instance or closure state set in @given and consumed in @when/@then instead of global variables, demonstrating Vedro's context sharing pattern.",
      "max_score": 10
    },
    {
      "name": "Event tracking",
      "description": "Each given/when/then appends to an events list (or similar) in order, and the final then-steps assert that sequence, showing proper use of Vedro step hooks to record execution order.",
      "max_score": 15
    }
  ]
}
tessl i tessl/pypi-vedro@1.14.0

tile.json