CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-vedro

Pragmatic Testing Framework for Python with BDD-style syntax and pluggable architecture

49

1.08x
Quality

Pending

Does it follow best practices?

Impact

49%

1.08x

Average score across 10 eval scenarios

SecuritybySnyk

Pending

The risk profile of this skill

Overview
Eval results
Files

criteria.jsonevals/scenario-1/

{
  "context": "Evaluates how the solution uses Vedro's seeding controls to achieve deterministic runs, scenario-level overrides, and explicit seed reporting for reproducibility. Focuses solely on correct application of run-level flags and @seed overrides rather than general code quality.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Run seed flag",
      "description": "Invokes Vedro with `--seed <value>` (or equivalent run arguments) so rerunning with the same seed yields identical scenario outputs.",
      "max_score": 25
    },
    {
      "name": "Generated seed reuse",
      "description": "When no seed is provided, enables `--show-seeds` or Seeder config to surface the generated run seed and demonstrates that rerunning with that seed reproduces the prior outputs.",
      "max_score": 15
    },
    {
      "name": "Scenario override",
      "description": "Marks at least one scenario with `@seed(<int>)` to give it a deterministic seed independent of the run-level seed.",
      "max_score": 25
    },
    {
      "name": "Override precedence",
      "description": "Shows that a scenario decorated with `@seed` keeps its override even when the run is started with `--seed`, producing output that differs from run-seeded scenarios while remaining repeatable.",
      "max_score": 15
    },
    {
      "name": "Seed reporting",
      "description": "Run output/report explicitly includes both the run seed and any scenario-level seed values (e.g., via `--show-seeds` or reporter extras) rather than keeping them implicit.",
      "max_score": 20
    }
  ]
}

tile.json