CtrlK
CommunityDocumentationLog inGet started
Tessl Logo

tessl/pypi-vedro

Pragmatic Testing Framework for Python with BDD-style syntax and pluggable architecture

49%

Overall

Evaluation49%

1.09x

Agent success when using this tile

Overview
Eval results
Files

rubric.jsonevals/scenario-7/

{
  "context": "Evaluates how the solution drives Vedro's CLI runner using vedro.cfg.py and the plugin registry. Checks that suites run through official commands, config wires plugins, and plugin toggling/listing is done via the package tools.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "vedro run",
      "description": "run_suite launches the suite through the `vedro run` CLI (or vedro.run entrypoint) and passes `--project-dir`/path arguments so execution relies on the framework runner instead of manual scenario execution.",
      "max_score": 25
    },
    {
      "name": "Config loading",
      "description": "Execution picks up settings from `project_dir/vedro.cfg.py`, using a Config subclass to define reporters and plugins via Config.plugins rather than hard-coding them in code.",
      "max_score": 20
    },
    {
      "name": "Plugin wiring",
      "description": "The audit plugin subclass extends `vedro.core.Plugin` and is registered through a `PluginConfig` (or added to Config.plugins) so it is visible to the CLI registry and runs when enabled.",
      "max_score": 20
    },
    {
      "name": "Enable/disable",
      "description": "Plugin state changes rely on `vedro plugin enable` / `vedro plugin disable` commands (or equivalent plugin manager toggles) and runs show the audit log when enabled and no log when disabled.",
      "max_score": 20
    },
    {
      "name": "Registry listing",
      "description": "Solution surfaces the output of `vedro plugin list` (or `vedro plugin`) and shows the audit plugin with correct enabled/disabled status after toggling.",
      "max_score": 15
    }
  ]
}
tessl i tessl/pypi-vedro@1.14.0

tile.json