0
# CLI and Execution
1
2
Command-line interface for running tests and managing plugins with extensive configuration options.
3
4
## Capabilities
5
6
### Primary Entry Point
7
8
Main function to run vedro tests programmatically or via command line.
9
10
```python { .api }
11
def run(*, plugins=None) -> None:
12
"""
13
Primary entry point to run vedro tests.
14
15
Args:
16
plugins: Deprecated parameter for plugin specification.
17
Use vedro.cfg.py configuration instead.
18
19
Raises:
20
DeprecationWarning: If plugins parameter is used
21
"""
22
```
23
24
#### Usage Example
25
26
```python
27
import vedro
28
29
# Run tests programmatically
30
vedro.run()
31
32
# The above is equivalent to running from command line:
33
# python -m vedro
34
# or just: vedro
35
```
36
37
### Command Line Interface
38
39
Vedro provides a comprehensive CLI with multiple commands and options.
40
41
#### Run Command (Default)
42
43
Execute test scenarios with various options and filters.
44
45
```bash
46
# Basic usage - runs all tests
47
vedro
48
49
# Explicit run command
50
vedro run
51
52
# Run with specific project directory
53
vedro run --project-dir /path/to/tests
54
55
# Run with plugin-specific arguments (plugin-dependent)
56
vedro run --seed 12345 # Set random seed
57
vedro run --order random # Random test order
58
vedro run --slice 1/4 # Run first quarter of tests
59
vedro run --last-failed # Rerun only failed tests
60
vedro run --dry-run # Show what would run without executing
61
vedro run --tags smoke,integration # Run tests with specific tags
62
vedro run --reporter rich # Use rich console reporter
63
vedro run --artifacts-dir artifacts/ # Specify artifacts directory
64
```
65
66
#### Version Command
67
68
Display the current vedro version.
69
70
```bash
71
vedro version
72
# Output: Vedro 1.14.3
73
```
74
75
#### Plugin Command
76
77
Manage plugins through the command line interface.
78
79
```bash
80
# List all available plugins
81
vedro plugin list
82
83
# Show popular plugins from registry
84
vedro plugin top
85
86
# Install and enable plugins
87
vedro plugin install vedro-reporter-html
88
vedro plugin install vedro-database-cleaner
89
90
# Enable/disable plugins
91
vedro plugin enable vedro-reporter-html
92
vedro plugin disable vedro-database-cleaner
93
94
# Multiple plugin operations
95
vedro plugin install plugin1 plugin2 plugin3
96
```
97
98
## CLI Architecture
99
100
### Command Structure
101
102
The CLI follows a modular command architecture with extensible argument parsing.
103
104
```python { .api }
105
# Base command interface (for custom commands)
106
class Command:
107
"""Abstract base class for CLI commands."""
108
109
def __init__(self, config: Type[Config], arg_parser: CommandArgumentParser): ...
110
async def run(self) -> None: ...
111
112
class CommandArgumentParser:
113
"""Enhanced argument parser for vedro commands."""
114
115
def add_argument(self, *args, **kwargs): ...
116
def parse_args(self): ...
117
def parse_known_args(self): ...
118
```
119
120
### Built-in Commands
121
122
Core commands provided by the vedro CLI system.
123
124
```python { .api }
125
class RunCommand(Command):
126
"""
127
Command for executing test scenarios.
128
129
Handles test discovery, plugin initialization, scenario execution,
130
and result reporting with extensive configuration options.
131
"""
132
133
async def run(self) -> None: ...
134
135
class VersionCommand(Command):
136
"""
137
Command for displaying version information.
138
139
Shows the current vedro version in a formatted output.
140
"""
141
142
async def run(self) -> None: ...
143
144
class PluginCommand(Command):
145
"""
146
Command for plugin management operations.
147
148
Supports listing, installing, enabling, and disabling plugins
149
through various subcommands.
150
"""
151
152
async def run(self) -> None: ...
153
```
154
155
## Usage Examples
156
157
### Basic Test Execution
158
159
```bash
160
# Run all tests in current directory
161
vedro
162
163
# Run tests with verbose output
164
vedro run --reporter rich
165
166
# Run tests in specific directory
167
vedro run --project-dir tests/
168
169
# Run with specific ordering
170
vedro run --order random --seed 42
171
172
# Run subset of tests
173
vedro run --slice 2/4 # Run second quarter of tests
174
```
175
176
### Advanced Test Filtering
177
178
```bash
179
# Run only tests with specific tags
180
vedro run --tags "smoke,critical"
181
182
# Exclude tests with certain tags
183
vedro run --exclude-tags "slow,flaky"
184
185
# Run only previously failed tests
186
vedro run --last-failed
187
188
# Run specific test files or patterns
189
vedro run scenarios/user_auth/
190
vedro run scenarios/test_login.py
191
```
192
193
### Plugin Management
194
195
```bash
196
# Install popular reporting plugin
197
vedro plugin install vedro-reporter-html
198
199
# Install multiple plugins at once
200
vedro plugin install vedro-screenshot-capture vedro-allure-reporter
201
202
# List installed plugins with status
203
vedro plugin list
204
205
# Enable a disabled plugin
206
vedro plugin enable vedro-reporter-html
207
208
# Disable a plugin temporarily
209
vedro plugin disable vedro-slow-tests
210
```
211
212
### Configuration and Environment
213
214
```bash
215
# Run with specific configuration file
216
vedro run --config custom_vedro.cfg.py
217
218
# Set environment variables
219
TEST_ENV=staging vedro run
220
221
# Run with different output directory
222
vedro run --artifacts-dir results/$(date +%Y%m%d_%H%M%S)
223
224
# Run with custom timeout
225
vedro run --timeout 300 # 5 minutes per scenario
226
```
227
228
### Development and Debugging
229
230
```bash
231
# Dry run - show what would execute without running
232
vedro run --dry-run
233
234
# Run with detailed debugging information
235
vedro run --verbose --debug
236
237
# Run with performance monitoring
238
vedro run --profile --performance-report
239
240
# Stop on first failure
241
vedro run --fail-fast
242
243
# Run with specific reporter for CI
244
CI=true vedro run --reporter silent --output-format junit
245
```
246
247
## Plugin Integration
248
249
### Plugin-provided CLI Arguments
250
251
Plugins can extend the CLI with custom arguments through the event system.
252
253
```python
254
from vedro.core import Plugin, PluginConfig
255
from vedro.events import ArgParseEvent
256
257
class CustomPlugin(Plugin):
258
"""Plugin that adds custom CLI arguments."""
259
260
def subscribe(self, dispatcher):
261
dispatcher.listen(ArgParseEvent, self.add_arguments)
262
263
def add_arguments(self, event: ArgParseEvent):
264
"""Add custom arguments to the CLI parser."""
265
parser = event.arg_parser
266
267
parser.add_argument(
268
"--custom-option",
269
help="Custom option provided by plugin",
270
default="default_value"
271
)
272
273
parser.add_argument(
274
"--enable-feature",
275
action="store_true",
276
help="Enable special feature"
277
)
278
279
parser.add_argument(
280
"--output-format",
281
choices=["json", "xml", "csv"],
282
default="json",
283
help="Output format for results"
284
)
285
286
class CustomPluginConfig(PluginConfig):
287
plugin = CustomPlugin
288
enabled = True
289
```
290
291
### Accessing CLI Arguments
292
293
Plugins can access parsed arguments through events.
294
295
```python
296
from vedro.events import ArgParsedEvent
297
298
class ArgumentConsumerPlugin(Plugin):
299
"""Plugin that uses CLI arguments."""
300
301
def __init__(self, config):
302
super().__init__(config)
303
self.args = None
304
305
def subscribe(self, dispatcher):
306
dispatcher.listen(ArgParsedEvent, self.handle_args)
307
308
def handle_args(self, event: ArgParsedEvent):
309
"""Process parsed CLI arguments."""
310
self.args = event.args
311
312
# Access custom arguments
313
if hasattr(self.args, 'custom_option'):
314
print(f"Custom option value: {self.args.custom_option}")
315
316
if hasattr(self.args, 'enable_feature') and self.args.enable_feature:
317
print("Special feature enabled!")
318
319
# Configure plugin behavior based on arguments
320
if hasattr(self.args, 'output_format'):
321
self.configure_output_format(self.args.output_format)
322
```
323
324
## Advanced CLI Patterns
325
326
### Environment-based Command Selection
327
328
Create wrapper scripts for different environments:
329
330
```bash
331
#!/bin/bash
332
# run_staging_tests.sh
333
334
export TEST_ENV=staging
335
export DATABASE_URL=postgresql://staging-db:5432/testdb
336
337
vedro run \
338
--tags "smoke,integration" \
339
--reporter html \
340
--artifacts-dir results/staging \
341
--timeout 600
342
```
343
344
```bash
345
#!/bin/bash
346
# run_ci_tests.sh
347
348
export CI=true
349
export TEST_ENV=ci
350
351
vedro run \
352
--reporter silent \
353
--output-format junit \
354
--artifacts-dir $CI_ARTIFACTS_DIR \
355
--fail-fast \
356
--last-failed
357
```
358
359
### Complex Test Selection
360
361
Use advanced filtering for complex test suites:
362
363
```bash
364
# Run performance tests only on weekends
365
if [ $(date +%u) -gt 5 ]; then
366
vedro run --tags performance --timeout 1800
367
else
368
vedro run --exclude-tags "performance,slow"
369
fi
370
371
# Run different test suites based on git branch
372
BRANCH=$(git branch --show-current)
373
case $BRANCH in
374
"main"|"master")
375
vedro run --tags "smoke,critical" --fail-fast
376
;;
377
"develop")
378
vedro run --exclude-tags "flaky"
379
;;
380
"feature/"*)
381
vedro run --tags "unit,integration" --slice 1/2
382
;;
383
*)
384
vedro run --dry-run
385
;;
386
esac
387
```
388
389
### Batch Operations
390
391
Run multiple test configurations in sequence:
392
393
```bash
394
#!/bin/bash
395
# comprehensive_test_run.sh
396
397
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
398
RESULTS_DIR="results/$TIMESTAMP"
399
400
# Create results directory
401
mkdir -p "$RESULTS_DIR"
402
403
# Run smoke tests first
404
echo "Running smoke tests..."
405
vedro run \
406
--tags smoke \
407
--reporter html \
408
--artifacts-dir "$RESULTS_DIR/smoke" \
409
--fail-fast
410
411
if [ $? -eq 0 ]; then
412
echo "Smoke tests passed, running full suite..."
413
414
# Run full test suite
415
vedro run \
416
--exclude-tags "performance,flaky" \
417
--reporter html \
418
--artifacts-dir "$RESULTS_DIR/full" \
419
--parallel 4
420
421
# Run performance tests if full suite passes
422
if [ $? -eq 0 ]; then
423
echo "Running performance tests..."
424
vedro run \
425
--tags performance \
426
--reporter json \
427
--artifacts-dir "$RESULTS_DIR/performance" \
428
--timeout 3600
429
fi
430
else
431
echo "Smoke tests failed, skipping full suite"
432
exit 1
433
fi
434
435
# Generate combined report
436
echo "Generating combined report..."
437
python generate_combined_report.py "$RESULTS_DIR"
438
```
439
440
### Integration with CI/CD
441
442
Example GitHub Actions workflow:
443
444
```yaml
445
name: Vedro Tests
446
447
on: [push, pull_request]
448
449
jobs:
450
test:
451
runs-on: ubuntu-latest
452
strategy:
453
matrix:
454
python-version: [3.8, 3.9, "3.10", "3.11"]
455
456
steps:
457
- uses: actions/checkout@v3
458
459
- name: Set up Python ${{ matrix.python-version }}
460
uses: actions/setup-python@v3
461
with:
462
python-version: ${{ matrix.python-version }}
463
464
- name: Install dependencies
465
run: |
466
pip install vedro
467
pip install -r requirements-test.txt
468
469
- name: Run smoke tests
470
run: |
471
vedro run \
472
--tags smoke \
473
--reporter silent \
474
--fail-fast
475
476
- name: Run full test suite
477
if: success()
478
run: |
479
vedro run \
480
--exclude-tags "slow,flaky" \
481
--reporter junit \
482
--artifacts-dir test-results
483
484
- name: Upload test results
485
if: always()
486
uses: actions/upload-artifact@v3
487
with:
488
name: test-results-${{ matrix.python-version }}
489
path: test-results/
490
```
491
492
## Configuration File Integration
493
494
### Project-level Configuration
495
496
The CLI automatically loads configuration from `vedro.cfg.py`:
497
498
```python
499
# vedro.cfg.py
500
import vedro
501
from pathlib import Path
502
503
class Config(vedro.Config):
504
"""Project-specific vedro configuration."""
505
506
# Default CLI behavior
507
default_tags = ["smoke", "integration"]
508
exclude_tags = ["flaky", "manual"]
509
510
# Output configuration
511
artifacts_dir = Path("test_artifacts")
512
513
class Plugins(vedro.Config.Plugins):
514
class RichReporter(vedro.Config.Plugins.RichReporter):
515
enabled = True
516
show_timings = True
517
518
class HtmlReporter(HtmlReporterConfig):
519
enabled = True
520
output_file = "test_report.html"
521
```
522
523
### User-level Configuration
524
525
Global configuration in `~/.vedro/config.py`:
526
527
```python
528
# ~/.vedro/config.py
529
import vedro
530
from pathlib import Path
531
import os
532
533
class GlobalConfig(vedro.Config):
534
"""User-specific vedro configuration."""
535
536
# User preferences
537
preferred_reporter = "rich"
538
539
class Plugins(vedro.Config.Plugins):
540
class LastFailed(vedro.Config.Plugins.LastFailed):
541
enabled = True
542
cache_file = Path.home() / ".vedro" / "last_failed"
543
544
class PerformanceMonitor(PerformanceMonitorConfig):
545
enabled = os.environ.get("VEDRO_PERF_MONITOR") == "true"
546
```