0
# Task Collections
1
2
Task collections provide hierarchical organization of tasks into namespaces with auto-discovery, configuration integration, and flexible task management for building complex CLI applications.
3
4
## Capabilities
5
6
### Collection Class
7
8
Main container for organizing tasks and sub-collections into hierarchical namespaces.
9
10
```python { .api }
11
class Collection:
12
"""
13
Hierarchical container for tasks and sub-collections.
14
15
Provides namespace management, task organization, auto-discovery from
16
modules, and configuration integration for building complex CLI applications.
17
18
Attributes:
19
- tasks (dict): Mapping of task names to Task objects
20
- collections (dict): Mapping of collection names to Collection objects
21
- default (str): Name of default task
22
- name (str): Collection name
23
- loaded_from (str): Path collection was loaded from
24
- auto_dash_names (bool): Convert underscores to dashes in task names
25
- _configuration (dict): Collection-specific configuration
26
"""
27
28
def __init__(self, *args, **kwargs):
29
"""
30
Initialize Collection.
31
32
Parameters:
33
- *args: Tasks or collections to add
34
- name (str, optional): Collection name
35
- default (str, optional): Default task name
36
- auto_dash_names (bool): Convert underscores to dashes
37
- **kwargs: Additional task name -> task mappings
38
"""
39
40
def add_task(self, task, name=None, aliases=None, default=None):
41
"""
42
Add task to collection.
43
44
Parameters:
45
- task (Task or callable): Task to add
46
- name (str, optional): Task name (defaults to task.name or function name)
47
- aliases (list, optional): Alternative names for task
48
- default (bool, optional): Whether this is the default task
49
50
Returns:
51
Task: Added task object
52
"""
53
54
def add_collection(self, coll, name=None):
55
"""
56
Add sub-collection to this collection.
57
58
Parameters:
59
- coll (Collection): Collection to add
60
- name (str, optional): Name for sub-collection
61
"""
62
63
@classmethod
64
def from_module(cls, module, name=None, config=None, loaded_from=None, auto_dash_names=True):
65
"""
66
Create collection from Python module.
67
68
Automatically discovers tasks (decorated functions) and sub-collections
69
in the given module and creates a Collection containing them.
70
71
Parameters:
72
- module: Python module to load from
73
- name (str, optional): Collection name
74
- config (Config, optional): Configuration object
75
- loaded_from (str, optional): Module file path
76
- auto_dash_names (bool): Convert underscores to dashes
77
78
Returns:
79
Collection: Collection loaded from module
80
"""
81
82
def subcollection_from_path(self, path):
83
"""
84
Get sub-collection from dotted path.
85
86
Parameters:
87
- path (str): Dotted path to sub-collection (e.g., 'deploy.staging')
88
89
Returns:
90
Collection: Sub-collection at path
91
92
Raises:
93
CollectionNotFound: If path not found
94
"""
95
96
def task_with_config(self, name, config):
97
"""
98
Get task with merged configuration.
99
100
Parameters:
101
- name (str): Task name
102
- config (Config): Base configuration
103
104
Returns:
105
Task: Task with merged configuration
106
"""
107
108
def to_contexts(self, config):
109
"""
110
Convert collection to parser contexts.
111
112
Parameters:
113
- config (Config): Configuration object
114
115
Returns:
116
list: List of ParserContext objects for CLI parsing
117
"""
118
119
def transform(self, name, config):
120
"""
121
Transform task name according to collection settings.
122
123
Parameters:
124
- name (str): Original task name
125
- config (Config): Configuration object
126
127
Returns:
128
str: Transformed task name
129
"""
130
131
def configuration(self, taskname=None):
132
"""
133
Get configuration for task or collection.
134
135
Parameters:
136
- taskname (str, optional): Specific task name
137
138
Returns:
139
dict: Configuration dictionary
140
"""
141
142
def configure(self, options):
143
"""
144
Configure collection with options.
145
146
Parameters:
147
- options (dict): Configuration options
148
"""
149
150
def serialized(self):
151
"""
152
Get serialized representation of collection.
153
154
Returns:
155
dict: Serialized collection data
156
"""
157
158
@property
159
def task_names(self):
160
"""
161
Get list of all task names in collection.
162
163
Returns:
164
list: Task names
165
"""
166
167
def __getitem__(self, name):
168
"""Get task or sub-collection by name."""
169
170
def __contains__(self, name):
171
"""Test if task or collection exists."""
172
173
def __iter__(self):
174
"""Iterate over task and collection names."""
175
```
176
177
### FilesystemLoader Class
178
179
Loads task collections from filesystem modules and packages.
180
181
```python { .api }
182
class FilesystemLoader:
183
"""
184
Loads Collections from filesystem.
185
186
Handles loading task collections from Python modules and packages
187
on the filesystem with support for nested discovery and import handling.
188
189
Attributes:
190
- start (str): Starting directory for collection discovery
191
"""
192
193
def __init__(self, start=None):
194
"""
195
Initialize FilesystemLoader.
196
197
Parameters:
198
- start (str, optional): Starting directory path
199
"""
200
201
def find(self, name):
202
"""
203
Find module for given collection name.
204
205
Parameters:
206
- name (str): Collection name to find
207
208
Returns:
209
str: Path to module file
210
211
Raises:
212
CollectionNotFound: If collection not found
213
"""
214
215
def load(self, name=None):
216
"""
217
Load collection by name.
218
219
Parameters:
220
- name (str, optional): Collection name to load
221
222
Returns:
223
Collection: Loaded collection
224
225
Raises:
226
CollectionNotFound: If collection not found
227
"""
228
229
@property
230
def start(self):
231
"""Starting directory for collection discovery."""
232
```
233
234
### Executor Class
235
236
Orchestrates task execution with dependency resolution and call management.
237
238
```python { .api }
239
class Executor:
240
"""
241
Task execution orchestrator.
242
243
Handles task execution order, dependency resolution, call deduplication,
244
and execution coordination for complex task workflows.
245
246
Attributes:
247
- collection (Collection): Task collection
248
- config (Config): Configuration object
249
- core (list): Core argument list
250
"""
251
252
def __init__(self, collection, config=None, core=None):
253
"""
254
Initialize Executor.
255
256
Parameters:
257
- collection (Collection): Task collection
258
- config (Config, optional): Configuration object
259
- core (list, optional): Core arguments
260
"""
261
262
def execute(self, *tasks):
263
"""
264
Execute tasks in proper order.
265
266
Parameters:
267
- *tasks: Task names or Call objects to execute
268
269
Returns:
270
dict: Execution results
271
"""
272
273
def normalize(self, tasks):
274
"""
275
Normalize task specifications to Call objects.
276
277
Parameters:
278
- tasks (list): Task specifications
279
280
Returns:
281
list: Normalized Call objects
282
"""
283
284
def dedupe(self, tasks):
285
"""
286
Remove duplicate task calls.
287
288
Parameters:
289
- tasks (list): List of Call objects
290
291
Returns:
292
list: Deduplicated Call objects
293
"""
294
295
def expand_calls(self, calls):
296
"""
297
Expand calls with pre/post task dependencies.
298
299
Parameters:
300
- calls (list): Call objects to expand
301
302
Returns:
303
list: Expanded calls with dependencies
304
"""
305
```
306
307
## Usage Examples
308
309
### Basic Collection Creation
310
311
```python
312
from invoke import Collection, task
313
314
@task
315
def hello(ctx, name='World'):
316
"""Say hello."""
317
print(f"Hello {name}!")
318
319
@task
320
def goodbye(ctx, name='World'):
321
"""Say goodbye."""
322
print(f"Goodbye {name}!")
323
324
# Create collection manually
325
ns = Collection()
326
ns.add_task(hello)
327
ns.add_task(goodbye, default=True) # Make goodbye the default task
328
329
# Or create from tasks directly
330
ns = Collection(hello, goodbye)
331
332
# Or use keyword arguments
333
ns = Collection(greet=hello, farewell=goodbye)
334
```
335
336
### Module-based Collections
337
338
Create a tasks module (`tasks.py`):
339
340
```python
341
# tasks.py
342
from invoke import task, Collection
343
344
@task
345
def clean(ctx):
346
"""Clean build artifacts."""
347
ctx.run("rm -rf build/ dist/ *.egg-info/")
348
349
@task
350
def build(ctx):
351
"""Build the project."""
352
ctx.run("python setup.py build")
353
354
@task
355
def test(ctx):
356
"""Run tests."""
357
ctx.run("python -m pytest")
358
359
# Optional: Create sub-collections
360
deploy = Collection('deploy')
361
362
@task
363
def staging(ctx):
364
"""Deploy to staging."""
365
ctx.run("deploy-staging.sh")
366
367
@task
368
def production(ctx):
369
"""Deploy to production."""
370
ctx.run("deploy-production.sh")
371
372
deploy.add_task(staging)
373
deploy.add_task(production)
374
375
# Main collection includes deploy sub-collection
376
ns = Collection(clean, build, test, deploy)
377
```
378
379
Load collection from module:
380
381
```python
382
from invoke import Collection
383
384
# Load collection from tasks.py module
385
import tasks
386
ns = Collection.from_module(tasks)
387
388
# Or load by module name
389
ns = Collection.from_module('tasks')
390
```
391
392
### Hierarchical Collections
393
394
```python
395
from invoke import Collection, task
396
397
# Database tasks
398
@task
399
def migrate(ctx):
400
"""Run database migrations."""
401
ctx.run("python manage.py migrate")
402
403
@task
404
def seed(ctx):
405
"""Seed database with test data."""
406
ctx.run("python manage.py loaddata fixtures.json")
407
408
db = Collection('db')
409
db.add_task(migrate)
410
db.add_task(seed)
411
412
# Deployment tasks
413
@task
414
def staging(ctx):
415
"""Deploy to staging."""
416
ctx.run("deploy-staging.sh")
417
418
@task
419
def production(ctx):
420
"""Deploy to production."""
421
ctx.run("deploy-production.sh")
422
423
deploy = Collection('deploy')
424
deploy.add_task(staging)
425
deploy.add_task(production)
426
427
# Main collection
428
ns = Collection(db, deploy)
429
430
# Usage:
431
# invoke db.migrate
432
# invoke db.seed
433
# invoke deploy.staging
434
# invoke deploy.production
435
```
436
437
### Collection Configuration
438
439
```python
440
from invoke import Collection, task, Config
441
442
@task
443
def serve(ctx):
444
"""Start development server."""
445
port = ctx.config.server.port
446
host = ctx.config.server.host
447
ctx.run(f"python manage.py runserver {host}:{port}")
448
449
# Create collection with configuration
450
config = Config({
451
'server': {
452
'host': 'localhost',
453
'port': 8000
454
}
455
})
456
457
ns = Collection(serve)
458
ns.configure({'server': {'port': 3000}}) # Override port
459
```
460
461
### Auto-discovery Collections
462
463
```python
464
# Directory structure:
465
# myproject/
466
# tasks/
467
# __init__.py
468
# build.py
469
# deploy.py
470
# test.py
471
472
# tasks/__init__.py
473
from invoke import Collection
474
from . import build, deploy, test
475
476
ns = Collection.from_module(build, name='build')
477
ns.add_collection(Collection.from_module(deploy, name='deploy'))
478
ns.add_collection(Collection.from_module(test, name='test'))
479
480
# tasks/build.py
481
from invoke import task
482
483
@task
484
def clean(ctx):
485
"""Clean build artifacts."""
486
ctx.run("rm -rf build/")
487
488
@task
489
def compile(ctx):
490
"""Compile source code."""
491
ctx.run("python setup.py build")
492
493
# Usage:
494
# invoke build.clean
495
# invoke build.compile
496
# invoke deploy.staging
497
# invoke test.unit
498
```
499
500
### Task Dependencies with Collections
501
502
```python
503
from invoke import Collection, task
504
505
@task
506
def clean(ctx):
507
"""Clean build directory."""
508
ctx.run("rm -rf build/")
509
510
@task
511
def compile(ctx):
512
"""Compile source code."""
513
ctx.run("gcc -o myapp src/*.c")
514
515
@task(pre=[clean, compile])
516
def package(ctx):
517
"""Package application."""
518
ctx.run("tar -czf myapp.tar.gz myapp")
519
520
@task(pre=[package])
521
def deploy(ctx):
522
"""Deploy packaged application."""
523
ctx.run("scp myapp.tar.gz server:/opt/")
524
525
ns = Collection(clean, compile, package, deploy)
526
527
# Running 'invoke deploy' will automatically run:
528
# 1. clean
529
# 2. compile
530
# 3. package
531
# 4. deploy
532
```
533
534
### Collection Introspection
535
536
```python
537
from invoke import Collection, task
538
539
@task
540
def hello(ctx):
541
pass
542
543
@task
544
def world(ctx):
545
pass
546
547
ns = Collection(hello, world)
548
549
# List all tasks
550
print(ns.task_names) # ['hello', 'world']
551
552
# Check if task exists
553
print('hello' in ns) # True
554
555
# Get task object
556
task_obj = ns['hello']
557
print(task_obj.name) # 'hello'
558
559
# Iterate over tasks
560
for name in ns:
561
task_obj = ns[name]
562
print(f"Task: {name}")
563
```
564
565
### Filesystem Loader Usage
566
567
```python
568
from invoke.loader import FilesystemLoader
569
570
# Create loader
571
loader = FilesystemLoader()
572
573
# Find task module
574
module_path = loader.find('tasks') # Looks for tasks.py or tasks/__init__.py
575
576
# Load collection
577
collection = loader.load('tasks')
578
579
# Use with custom start directory
580
loader = FilesystemLoader(start='/path/to/project')
581
collection = loader.load('deployment_tasks')
582
```