0
# Stacks
1
2
Stack and stack component classes for configuring ML infrastructure. A ZenML stack represents a complete infrastructure configuration combining an orchestrator, artifact store, and optional components like container registries, experiment trackers, and step operators.
3
4
## Capabilities
5
6
### Stack Class
7
8
```python { .api }
9
class Stack:
10
"""
11
Complete ZenML stack configuration.
12
13
A stack combines multiple stack components to provide
14
a complete ML infrastructure configuration.
15
16
Attributes:
17
- name: Stack name
18
- components: Dict mapping component types to component instances
19
- id: Stack UUID
20
"""
21
22
@property
23
def orchestrator(self):
24
"""Get the orchestrator component."""
25
26
@property
27
def artifact_store(self):
28
"""Get the artifact store component."""
29
30
@property
31
def container_registry(self):
32
"""Get the container registry component (optional)."""
33
34
@property
35
def step_operator(self):
36
"""Get the step operator component (optional)."""
37
38
@property
39
def experiment_tracker(self):
40
"""Get the experiment tracker component (optional)."""
41
42
@property
43
def model_deployer(self):
44
"""Get the model deployer component (optional)."""
45
46
@property
47
def feature_store(self):
48
"""Get the feature store component (optional)."""
49
50
@property
51
def model_registry(self):
52
"""Get the model registry component (optional)."""
53
54
@property
55
def alerter(self):
56
"""Get the alerter component (optional)."""
57
58
@property
59
def annotator(self):
60
"""Get the annotator component (optional)."""
61
62
@property
63
def data_validator(self):
64
"""Get the data validator component (optional)."""
65
66
@property
67
def image_builder(self):
68
"""Get the image builder component (optional)."""
69
70
@property
71
def deployer(self):
72
"""Get the deployer component (optional)."""
73
74
def dict(self) -> dict:
75
"""Get stack configuration as dict."""
76
```
77
78
Import from:
79
80
```python
81
from zenml.stack import Stack
82
```
83
84
### Stack Component
85
86
```python { .api }
87
class StackComponent:
88
"""
89
Base class for stack components.
90
91
All stack components (orchestrators, artifact stores, etc.)
92
inherit from this class.
93
94
Attributes:
95
- name: Component name
96
- id: Component UUID
97
- type: Component type (from StackComponentType enum)
98
- flavor: Component flavor
99
- config: Component configuration
100
"""
101
102
@property
103
def name(self) -> str:
104
"""Get component name."""
105
106
@property
107
def type(self) -> str:
108
"""Get component type."""
109
110
@property
111
def flavor(self) -> str:
112
"""Get component flavor."""
113
114
@property
115
def config(self):
116
"""Get component configuration."""
117
```
118
119
Import from:
120
121
```python
122
from zenml.stack import StackComponent
123
```
124
125
### Stack Component Config
126
127
```python { .api }
128
class StackComponentConfig:
129
"""
130
Base configuration for stack components.
131
132
All stack component configurations inherit from this class.
133
Configuration classes are Pydantic models with validation.
134
"""
135
```
136
137
Import from:
138
139
```python
140
from zenml.stack import StackComponentConfig
141
```
142
143
### Flavor
144
145
```python { .api }
146
class Flavor:
147
"""
148
Flavor of a stack component.
149
150
Flavors define different implementations of stack components.
151
For example, the artifact_store component has flavors like
152
'local', 's3', 'gcs', 'azure'.
153
154
Attributes:
155
- name: Flavor name
156
- type: Component type
157
- config_class: Configuration class
158
- implementation_class: Implementation class
159
"""
160
161
@property
162
def name(self) -> str:
163
"""Get flavor name."""
164
165
@property
166
def type(self) -> str:
167
"""Get component type."""
168
169
@property
170
def config_class(self) -> type:
171
"""Get configuration class."""
172
173
@property
174
def implementation_class(self) -> type:
175
"""Get implementation class."""
176
```
177
178
Import from:
179
180
```python
181
from zenml.stack import Flavor
182
```
183
184
### Stack Validator
185
186
```python { .api }
187
class StackValidator:
188
"""
189
Validator for stack configurations.
190
191
Validates that stack components are compatible with each other
192
and that all required components are present.
193
"""
194
195
def validate(self, stack: Stack):
196
"""
197
Validate stack configuration.
198
199
Parameters:
200
- stack: Stack to validate
201
202
Raises:
203
StackValidationError: If stack configuration is invalid
204
"""
205
```
206
207
Import from:
208
209
```python
210
from zenml.stack import StackValidator
211
```
212
213
### Stack Component Type Enum
214
215
```python { .api }
216
class StackComponentType(str, Enum):
217
"""
218
Stack component types.
219
220
Values:
221
- ORCHESTRATOR: Pipeline orchestration (required)
222
- ARTIFACT_STORE: Artifact storage (required)
223
- CONTAINER_REGISTRY: Container image registry
224
- IMAGE_BUILDER: Container image builder
225
- STEP_OPERATOR: Remote step execution
226
- EXPERIMENT_TRACKER: Experiment tracking
227
- MODEL_DEPLOYER: Model deployment
228
- FEATURE_STORE: Feature store
229
- MODEL_REGISTRY: Model registry
230
- DATA_VALIDATOR: Data validation
231
- ALERTER: Alerting/notifications
232
- ANNOTATOR: Data annotation
233
- DEPLOYER: Pipeline deployment
234
"""
235
ORCHESTRATOR = "orchestrator"
236
ARTIFACT_STORE = "artifact_store"
237
CONTAINER_REGISTRY = "container_registry"
238
IMAGE_BUILDER = "image_builder"
239
STEP_OPERATOR = "step_operator"
240
EXPERIMENT_TRACKER = "experiment_tracker"
241
MODEL_DEPLOYER = "model_deployer"
242
FEATURE_STORE = "feature_store"
243
MODEL_REGISTRY = "model_registry"
244
DATA_VALIDATOR = "data_validator"
245
ALERTER = "alerter"
246
ANNOTATOR = "annotator"
247
DEPLOYER = "deployer"
248
```
249
250
Import from:
251
252
```python
253
from zenml.enums import StackComponentType
254
```
255
256
## Usage Examples
257
258
### Accessing Stack Components
259
260
```python
261
from zenml.client import Client
262
263
client = Client()
264
stack = client.active_stack
265
266
# Access required components
267
print(f"Orchestrator: {stack.orchestrator.name} ({stack.orchestrator.flavor})")
268
print(f"Artifact Store: {stack.artifact_store.name} ({stack.artifact_store.flavor})")
269
270
# Access optional components
271
if stack.container_registry:
272
print(f"Container Registry: {stack.container_registry.name}")
273
274
if stack.experiment_tracker:
275
print(f"Experiment Tracker: {stack.experiment_tracker.name}")
276
277
if stack.step_operator:
278
print(f"Step Operator: {stack.step_operator.name}")
279
```
280
281
### Creating a Stack via Client
282
283
```python
284
from zenml.client import Client
285
286
client = Client()
287
288
# Create components first
289
client.create_stack_component(
290
name="local_orchestrator",
291
flavor="local",
292
component_type="orchestrator",
293
configuration={}
294
)
295
296
client.create_stack_component(
297
name="local_artifact_store",
298
flavor="local",
299
component_type="artifact_store",
300
configuration={"path": "/tmp/zenml-artifacts"}
301
)
302
303
# Create stack with components
304
stack = client.create_stack(
305
name="local_stack",
306
components={
307
"orchestrator": "local_orchestrator",
308
"artifact_store": "local_artifact_store"
309
},
310
description="Local development stack"
311
)
312
313
# Activate the stack
314
client.activate_stack("local_stack")
315
```
316
317
### Creating Cloud Stack
318
319
```python
320
from zenml.client import Client
321
322
client = Client()
323
324
# Create S3 artifact store
325
client.create_stack_component(
326
name="s3_store",
327
flavor="s3",
328
component_type="artifact_store",
329
configuration={
330
"path": "s3://my-zenml-bucket/artifacts",
331
"region": "us-east-1"
332
}
333
)
334
335
# Create SageMaker orchestrator
336
client.create_stack_component(
337
name="sagemaker_orchestrator",
338
flavor="sagemaker",
339
component_type="orchestrator",
340
configuration={
341
"execution_role": "arn:aws:iam::123456789:role/SageMaker",
342
"region": "us-east-1"
343
}
344
)
345
346
# Create ECR container registry
347
client.create_stack_component(
348
name="ecr_registry",
349
flavor="aws",
350
component_type="container_registry",
351
configuration={
352
"uri": "123456789.dkr.ecr.us-east-1.amazonaws.com",
353
"region": "us-east-1"
354
}
355
)
356
357
# Create stack
358
aws_stack = client.create_stack(
359
name="aws_production",
360
components={
361
"orchestrator": "sagemaker_orchestrator",
362
"artifact_store": "s3_store",
363
"container_registry": "ecr_registry"
364
},
365
description="AWS production stack"
366
)
367
```
368
369
### Stack with Experiment Tracking
370
371
```python
372
from zenml.client import Client
373
374
client = Client()
375
376
# Create MLflow experiment tracker
377
client.create_stack_component(
378
name="mlflow_tracker",
379
flavor="mlflow",
380
component_type="experiment_tracker",
381
configuration={
382
"tracking_uri": "http://mlflow-server:5000",
383
"tracking_username": "admin",
384
"tracking_password": "password"
385
}
386
)
387
388
# Add to stack
389
client.update_stack(
390
name_or_id="local_stack",
391
components={
392
"orchestrator": "local_orchestrator",
393
"artifact_store": "local_artifact_store",
394
"experiment_tracker": "mlflow_tracker"
395
}
396
)
397
```
398
399
### Listing Available Flavors
400
401
```python
402
from zenml.client import Client
403
from zenml.enums import StackComponentType
404
405
client = Client()
406
407
# List all orchestrator flavors
408
orchestrator_flavors = client.get_flavors_by_type(
409
component_type=StackComponentType.ORCHESTRATOR
410
)
411
412
print("Available orchestrator flavors:")
413
for flavor in orchestrator_flavors:
414
print(f" - {flavor.name}: {flavor.description}")
415
416
# List all artifact store flavors
417
store_flavors = client.get_flavors_by_type(
418
component_type=StackComponentType.ARTIFACT_STORE
419
)
420
421
print("\nAvailable artifact store flavors:")
422
for flavor in store_flavors:
423
print(f" - {flavor.name}: {flavor.description}")
424
```
425
426
### Stack Component Configuration
427
428
```python
429
from zenml.client import Client
430
431
client = Client()
432
433
# Get component configuration
434
component = client.get_stack_component(
435
name_or_id="s3_store",
436
component_type="artifact_store"
437
)
438
439
print(f"Component: {component.name}")
440
print(f"Flavor: {component.flavor}")
441
print(f"Configuration: {component.configuration}")
442
443
# Update component configuration
444
client.update_stack_component(
445
name_or_id="s3_store",
446
component_type="artifact_store",
447
updated_configuration={
448
"path": "s3://my-new-bucket/artifacts",
449
"region": "us-west-2"
450
}
451
)
452
```
453
454
### Custom Stack Component
455
456
```python
457
from zenml.stack import StackComponent, StackComponentConfig
458
from zenml.enums import StackComponentType
459
460
class MyCustomConfig(StackComponentConfig):
461
"""Custom component configuration."""
462
api_key: str
463
endpoint: str
464
465
class MyCustomComponent(StackComponent):
466
"""Custom stack component implementation."""
467
468
@property
469
def config(self) -> MyCustomConfig:
470
"""Get typed configuration."""
471
return self._config
472
473
def custom_method(self):
474
"""Custom functionality."""
475
print(f"Connecting to {self.config.endpoint}")
476
```
477