0
# Compute Services
1
2
The compute service provides a unified interface for managing virtual machines, instances, and compute resources across 60+ cloud providers including AWS EC2, Azure, Google Compute Engine, Rackspace, Linode, DigitalOcean, and many others.
3
4
## Providers
5
6
```python { .api }
7
from libcloud.compute.types import Provider
8
9
class Provider:
10
"""Enumeration of supported compute providers"""
11
# Major cloud providers
12
AZURE = 'azure'
13
AZURE_ARM = 'azure_arm'
14
EC2 = 'ec2'
15
GCE = 'gce'
16
17
# Other providers (alphabetically)
18
ABIQUO = 'abiquo'
19
ALIYUN_ECS = 'aliyun_ecs'
20
AURORACOMPUTE = 'aurora_compute'
21
BRIGHTBOX = 'brightbox'
22
CLOUDSCALE = 'cloudscale'
23
CLOUDSTACK = 'cloudstack'
24
DIGITAL_OCEAN = 'digitalocean'
25
DIMENSIONDATA = 'dimensiondata'
26
EQUINIXMETAL = 'equinixmetal'
27
EXOSCALE = 'exoscale'
28
GRIDSCALE = 'gridscale'
29
IBM = 'ibm'
30
IKOULA = 'ikoula'
31
KAMATERA = 'kamatera'
32
KTUCLOUD = 'ktucloud'
33
KUBEVIRT = 'kubevirt'
34
LIBVIRT = 'libvirt'
35
LINODE = 'linode'
36
MAXIHOST = 'maxihost'
37
NIMBUS = 'nimbus'
38
NINEFOLD = 'ninefold'
39
NTTCIS = 'nttcis'
40
OPENNEBULA = 'opennebula'
41
OPENSTACK = 'openstack'
42
OUTSCALE = 'outscale'
43
OVH = 'ovh'
44
RACKSPACE = 'rackspace'
45
SCALEWAY = 'scaleway'
46
UPCLOUD = 'upcloud'
47
VCLOUD = 'vcloud'
48
VSPHERE = 'vsphere'
49
VULTR = 'vultr'
50
# ... and 20+ more providers
51
```
52
53
## Driver Factory
54
55
```python { .api }
56
from libcloud.compute.providers import get_driver
57
58
def get_driver(provider: Provider) -> type[NodeDriver]
59
```
60
61
Get the driver class for a specific compute provider.
62
63
**Parameters:**
64
- `provider`: Provider identifier from the Provider enum
65
66
**Returns:**
67
- Driver class for the specified provider
68
69
**Example:**
70
```python
71
from libcloud.compute.types import Provider
72
from libcloud.compute.providers import get_driver
73
74
# Get EC2 driver class
75
cls = get_driver(Provider.EC2)
76
77
# Initialize driver with credentials
78
driver = cls('access_key', 'secret_key', region='us-east-1')
79
```
80
81
## Core Classes
82
83
### NodeDriver
84
85
```python { .api }
86
class NodeDriver(BaseDriver):
87
"""Base class for all compute drivers"""
88
89
def list_nodes(self) -> List[Node]
90
def get_node(self, node_id: str) -> Node
91
def create_node(self, name: str, size: NodeSize, image: NodeImage, location: NodeLocation = None, **kwargs) -> Node
92
def reboot_node(self, node: Node) -> bool
93
def destroy_node(self, node: Node) -> bool
94
def list_images(self, location: NodeLocation = None, ex_only_active: bool = True) -> List[NodeImage]
95
def list_sizes(self, location: NodeLocation = None) -> List[NodeSize]
96
def list_locations(self) -> List[NodeLocation]
97
def deploy_node(self, deploy: Deployment, ssh_username: str = "root",
98
ssh_port: int = 22, ssh_timeout: int = 10, timeout: int = 300,
99
**create_node_kwargs) -> Node
100
def wait_until_running(self, nodes: List[Node], wait_period: float = 5, timeout: int = 600,
101
ssh_interface: str = "public_ips", force_ipv4: bool = True) -> List[Tuple[Node, List[str]]]
102
```
103
104
Base class that all compute drivers inherit from. Provides methods for managing virtual machines and instances.
105
106
**Key Methods:**
107
108
- `list_nodes()`: List all nodes/instances in the account
109
- `create_node()`: Create a new virtual machine instance
110
- `destroy_node()`: Terminate/delete a node
111
- `reboot_node()`: Restart a node
112
- `list_images()`: List available OS images/templates
113
- `list_sizes()`: List available instance sizes/flavors
114
- `list_locations()`: List available regions/datacenters
115
116
### Node
117
118
```python { .api }
119
class Node:
120
"""Represents a virtual machine/compute instance"""
121
122
id: str
123
name: str
124
state: NodeState
125
public_ips: List[str]
126
private_ips: List[str]
127
driver: NodeDriver
128
size: NodeSize
129
image: NodeImage
130
extra: Dict[str, Any]
131
created_at: datetime
132
133
def reboot(self) -> bool
134
def destroy(self) -> bool
135
def get_uuid(self) -> str
136
```
137
138
Represents a virtual machine or compute instance.
139
140
**Properties:**
141
- `id`: Unique identifier for the node
142
- `name`: Human-readable name
143
- `state`: Current state (running, stopped, pending, etc.)
144
- `public_ips`: List of public IP addresses
145
- `private_ips`: List of private IP addresses
146
- `size`: Instance size/flavor information
147
- `image`: OS image information
148
- `extra`: Provider-specific metadata
149
150
### NodeSize
151
152
```python { .api }
153
class NodeSize:
154
"""Represents an instance size/flavor"""
155
156
id: str
157
name: str
158
ram: int
159
disk: int
160
bandwidth: int
161
price: float
162
driver: NodeDriver
163
extra: Dict[str, Any]
164
```
165
166
Represents available instance sizes/flavors with their specifications.
167
168
**Properties:**
169
- `ram`: Memory in MB
170
- `disk`: Disk space in GB
171
- `bandwidth`: Network bandwidth
172
- `price`: Hourly cost
173
174
### NodeImage
175
176
```python { .api }
177
class NodeImage:
178
"""Represents an OS image/template"""
179
180
id: str
181
name: str
182
driver: NodeDriver
183
extra: Dict[str, Any]
184
```
185
186
Represents an operating system image or template that can be used to create instances.
187
188
### NodeLocation
189
190
```python { .api }
191
class NodeLocation:
192
"""Represents a geographic location/region"""
193
194
id: str
195
name: str
196
country: str
197
availability_zone: str
198
driver: NodeDriver
199
extra: Dict[str, Any]
200
```
201
202
Represents a geographic location, region, or availability zone where instances can be created.
203
204
### NodeState
205
206
```python { .api }
207
class NodeState:
208
"""Node lifecycle states"""
209
RUNNING = 0
210
REBOOTING = 1
211
TERMINATED = 2
212
PENDING = 3
213
UNKNOWN = 4
214
STOPPED = 5
215
SUSPENDED = 6
216
ERROR = 7
217
PAUSED = 8
218
RECONFIGURING = 9
219
MIGRATING = 10
220
STARTING = 11
221
STOPPING = 12
222
```
223
224
Enumeration of possible node states during their lifecycle.
225
226
## Authentication
227
228
### NodeAuth Classes
229
230
```python { .api }
231
class NodeAuthSSHKey:
232
"""SSH key authentication"""
233
def __init__(self, pubkey: str)
234
235
class NodeAuthPassword:
236
"""Password authentication"""
237
def __init__(self, password: str)
238
```
239
240
Authentication methods for accessing created nodes.
241
242
**Example:**
243
```python
244
from libcloud.compute.base import NodeAuthSSHKey, NodeAuthPassword
245
246
# SSH key authentication
247
ssh_key = NodeAuthSSHKey(open('~/.ssh/id_rsa.pub').read())
248
249
# Password authentication
250
password_auth = NodeAuthPassword('my_secure_password')
251
252
# Use in node creation
253
node = driver.create_node(
254
name='my-server',
255
size=sizes[0],
256
image=images[0],
257
auth=ssh_key
258
)
259
```
260
261
## Storage Volumes
262
263
### StorageVolume
264
265
```python { .api }
266
class StorageVolume(UuidMixin):
267
"""Represents a storage volume"""
268
269
def __init__(self, id: str, name: str, size: int, driver: NodeDriver,
270
state: StorageVolumeState = None, extra: Dict = None) -> None
271
272
# Instance methods
273
def list_snapshots(self) -> List[VolumeSnapshot]
274
def attach(self, node: Node, device: str = None) -> bool
275
def detach(self) -> bool
276
def snapshot(self, name: str) -> VolumeSnapshot
277
def destroy(self) -> bool
278
279
# Attributes
280
id: str
281
name: str
282
size: int
283
state: StorageVolumeState
284
driver: NodeDriver
285
extra: Dict[str, Any]
286
```
287
288
### VolumeSnapshot
289
290
```python { .api }
291
class VolumeSnapshot:
292
"""Represents a volume snapshot"""
293
294
def __init__(self, id: str, driver: NodeDriver, size: int = None,
295
extra: Dict = None, created: datetime = None,
296
state: StorageVolumeState = None, name: str = None) -> None
297
298
def destroy(self) -> bool
299
300
# Attributes
301
id: str
302
name: str
303
size: int
304
driver: NodeDriver
305
state: StorageVolumeState
306
created: datetime
307
extra: Dict[str, Any]
308
```
309
310
### Volume Management Methods
311
312
```python { .api }
313
class NodeDriver:
314
def list_volumes(self) -> List[StorageVolume]
315
def create_volume(self, size: int, name: str, location: NodeLocation = None, snapshot: VolumeSnapshot = None) -> StorageVolume
316
def destroy_volume(self, volume: StorageVolume) -> bool
317
def attach_volume(self, node: Node, volume: StorageVolume, device: str = None) -> bool
318
def detach_volume(self, volume: StorageVolume) -> bool
319
def list_volume_snapshots(self, volume: StorageVolume) -> List[VolumeSnapshot]
320
def create_volume_snapshot(self, volume: StorageVolume, name: str = None) -> VolumeSnapshot
321
def destroy_volume_snapshot(self, snapshot: VolumeSnapshot) -> bool
322
```
323
324
## SSH Key Management
325
326
### KeyPair
327
328
```python { .api }
329
class KeyPair:
330
"""Represents an SSH key pair"""
331
332
name: str
333
fingerprint: str
334
public_key: str
335
private_key: str
336
driver: NodeDriver
337
extra: Dict[str, Any]
338
```
339
340
### Key Management Methods
341
342
```python { .api }
343
class NodeDriver:
344
def list_key_pairs(self) -> List[KeyPair]
345
def create_key_pair(self, name: str) -> KeyPair
346
def import_key_pair_from_string(self, name: str, key_material: str) -> KeyPair
347
def import_key_pair_from_file(self, name: str, key_file_path: str) -> KeyPair
348
def delete_key_pair(self, key_pair: KeyPair) -> bool
349
```
350
351
## SSH Client Support
352
353
```python { .api }
354
from libcloud.compute.ssh import SSHClient
355
356
class SSHClient:
357
"""SSH client for connecting to nodes"""
358
359
def __init__(self, hostname: str, port: int = 22, username: str = 'root', password: str = None, key: str = None)
360
def connect(self) -> bool
361
def run(self, cmd: str) -> Tuple[str, str, int]
362
def put(self, path: str, contents: str, chmod: int = 755) -> bool
363
def delete(self, path: str) -> bool
364
def close(self) -> bool
365
```
366
367
Provides SSH connectivity to created nodes for remote command execution.
368
369
**Example:**
370
```python
371
from libcloud.compute.ssh import SSHClient
372
373
# Connect via SSH key
374
client = SSHClient(
375
hostname=node.public_ips[0],
376
username='ubuntu',
377
key='/path/to/private_key'
378
)
379
380
if client.connect():
381
stdout, stderr, exit_code = client.run('ls -la')
382
print(f"Command output: {stdout}")
383
client.close()
384
```
385
386
## Deployment Classes
387
388
Deployment classes provide automated server provisioning and configuration capabilities. They work with the `deploy_node()` method to set up servers after creation.
389
390
### Base Deployment Class
391
392
```python { .api }
393
from libcloud.compute.deployment import Deployment
394
395
class Deployment:
396
"""Base class for all deployment tasks"""
397
398
def run(self, node: Node, client: BaseSSHClient) -> Node
399
```
400
401
Abstract base class that all deployment steps inherit from.
402
403
### SSH Key Deployment
404
405
```python { .api }
406
from libcloud.compute.deployment import SSHKeyDeployment
407
408
class SSHKeyDeployment(Deployment):
409
"""Installs SSH public key for authentication"""
410
411
def __init__(self, key: Union[str, IO]) -> None
412
def run(self, node: Node, client: BaseSSHClient) -> Node
413
```
414
415
Installs a public SSH key into the server's authorized_keys file.
416
417
**Parameters:**
418
- `key`: SSH public key content as string or file object
419
420
**Example:**
421
```python
422
from libcloud.compute.deployment import SSHKeyDeployment
423
424
# From string
425
key_deploy = SSHKeyDeployment("ssh-rsa AAAAB3NzaC1yc2E...")
426
427
# From file
428
with open('/path/to/key.pub') as fp:
429
key_deploy = SSHKeyDeployment(fp)
430
```
431
432
### File Deployment
433
434
```python { .api }
435
from libcloud.compute.deployment import FileDeployment
436
437
class FileDeployment(Deployment):
438
"""Transfers files to the server"""
439
440
def __init__(self, source: str, target: str) -> None
441
def run(self, node: Node, client: BaseSSHClient) -> Node
442
```
443
444
Uploads a local file to the remote server.
445
446
**Parameters:**
447
- `source`: Local file path
448
- `target`: Remote destination path
449
450
**Example:**
451
```python
452
from libcloud.compute.deployment import FileDeployment
453
454
# Transfer configuration file
455
config_deploy = FileDeployment("/local/app.conf", "/etc/app/app.conf")
456
```
457
458
### Script Deployment
459
460
```python { .api }
461
from libcloud.compute.deployment import ScriptDeployment
462
463
class ScriptDeployment(Deployment):
464
"""Executes shell scripts on the server"""
465
466
def __init__(self, script: str, args: List[str] = None, name: str = None,
467
delete: bool = False, timeout: float = None) -> None
468
def run(self, node: Node, client: BaseSSHClient) -> Node
469
470
# Available after execution
471
stdout: str
472
stderr: str
473
exit_status: int
474
```
475
476
Uploads and executes a shell script on the server.
477
478
**Parameters:**
479
- `script`: Script content as string
480
- `args`: Optional command line arguments
481
- `name`: Optional script filename (random if not specified)
482
- `delete`: Whether to delete script after execution (default: False)
483
- `timeout`: Optional execution timeout in seconds
484
485
**Example:**
486
```python
487
from libcloud.compute.deployment import ScriptDeployment
488
489
# Basic script
490
install_script = ScriptDeployment('''#!/bin/bash
491
apt-get update
492
apt-get install -y nginx
493
systemctl start nginx
494
''')
495
496
# Script with arguments and timeout
497
config_script = ScriptDeployment(
498
script="#!/bin/bash\necho $1 > /etc/hostname",
499
args=["web-server"],
500
delete=True,
501
timeout=30
502
)
503
```
504
505
### Script File Deployment
506
507
```python { .api }
508
from libcloud.compute.deployment import ScriptFileDeployment
509
510
class ScriptFileDeployment(ScriptDeployment):
511
"""Executes shell scripts from local files"""
512
513
def __init__(self, script_file: str, args: List[str] = None, name: str = None,
514
delete: bool = False, timeout: float = None) -> None
515
```
516
517
Uploads and executes a shell script from a local file.
518
519
**Parameters:**
520
- `script_file`: Path to local script file
521
- `args`: Optional command line arguments
522
- `name`: Optional remote script filename
523
- `delete`: Whether to delete script after execution
524
- `timeout`: Optional execution timeout in seconds
525
526
**Example:**
527
```python
528
from libcloud.compute.deployment import ScriptFileDeployment
529
530
# Execute local installation script
531
install_deploy = ScriptFileDeployment(
532
script_file="/local/scripts/install.sh",
533
args=["--verbose", "--config=/etc/app.conf"],
534
delete=True,
535
timeout=300
536
)
537
```
538
539
### Multi-Step Deployment
540
541
```python { .api }
542
from libcloud.compute.deployment import MultiStepDeployment
543
544
class MultiStepDeployment(Deployment):
545
"""Chains multiple deployment steps together"""
546
547
def __init__(self, add: Union[Deployment, List[Deployment]] = None) -> None
548
def add(self, add: Union[Deployment, List[Deployment]]) -> None
549
def run(self, node: Node, client: BaseSSHClient) -> Node
550
551
steps: List[Deployment]
552
```
553
554
Executes multiple deployment steps in sequence.
555
556
**Parameters:**
557
- `add`: Initial deployment step(s) to add
558
559
**Example:**
560
```python
561
from libcloud.compute.deployment import (
562
SSHKeyDeployment, FileDeployment, ScriptDeployment, MultiStepDeployment
563
)
564
565
# Create individual steps
566
key_step = SSHKeyDeployment(public_key)
567
config_step = FileDeployment("/local/nginx.conf", "/etc/nginx/nginx.conf")
568
install_step = ScriptDeployment("apt-get update && apt-get install -y nginx")
569
570
# Combine into multi-step deployment
571
deploy = MultiStepDeployment([key_step, config_step, install_step])
572
573
# Or build incrementally
574
deploy = MultiStepDeployment()
575
deploy.add(key_step)
576
deploy.add(config_step)
577
deploy.add(install_step)
578
```
579
580
## Usage Examples
581
582
### Basic Node Management
583
584
```python
585
from libcloud.compute.types import Provider
586
from libcloud.compute.providers import get_driver
587
588
# Initialize driver
589
cls = get_driver(Provider.EC2)
590
driver = cls('access_key', 'secret_key', region='us-east-1')
591
592
# List existing nodes
593
nodes = driver.list_nodes()
594
for node in nodes:
595
print(f"Node: {node.name}, State: {node.state}, IPs: {node.public_ips}")
596
597
# Get available resources
598
sizes = driver.list_sizes()
599
images = driver.list_images()
600
locations = driver.list_locations()
601
602
print(f"Available sizes: {len(sizes)}")
603
print(f"Available images: {len(images)}")
604
print(f"Available locations: {len(locations)}")
605
```
606
607
### Creating and Managing Nodes
608
609
```python
610
from libcloud.compute.base import NodeAuthSSHKey
611
612
# Create SSH key authentication
613
ssh_key = NodeAuthSSHKey(open('~/.ssh/id_rsa.pub').read())
614
615
# Create a new node
616
node = driver.create_node(
617
name='web-server-01',
618
size=sizes[0], # t2.micro equivalent
619
image=images[0], # Ubuntu 20.04 LTS
620
location=locations[0], # us-east-1a
621
auth=ssh_key,
622
ex_security_groups=['web-servers'],
623
ex_userdata='#!/bin/bash\napt-get update'
624
)
625
626
print(f"Created node: {node.name} ({node.id})")
627
628
# Wait for node to be running
629
nodes_running = driver.wait_until_running([node])
630
print(f"Node is now running: {nodes_running[0][0].state}")
631
632
# Reboot the node
633
success = driver.reboot_node(node)
634
print(f"Reboot initiated: {success}")
635
636
# Destroy the node when done
637
success = driver.destroy_node(node)
638
print(f"Node destroyed: {success}")
639
```
640
641
### Working with Multiple Providers
642
643
```python
644
from libcloud.compute.types import Provider
645
from libcloud.compute.providers import get_driver
646
647
# Configure multiple providers
648
providers_config = {
649
'aws': {
650
'driver': get_driver(Provider.EC2),
651
'credentials': ('aws_access_key', 'aws_secret_key'),
652
'region': 'us-east-1'
653
},
654
'gce': {
655
'driver': get_driver(Provider.GCE),
656
'credentials': ('service_account_email', 'key_file_path'),
657
'project': 'my-project'
658
}
659
}
660
661
# Initialize drivers for each provider
662
drivers = {}
663
for name, config in providers_config.items():
664
cls = config['driver']
665
if name == 'aws':
666
drivers[name] = cls(*config['credentials'], region=config['region'])
667
elif name == 'gce':
668
drivers[name] = cls(*config['credentials'], project=config['project'])
669
670
# List nodes across all providers
671
all_nodes = {}
672
for provider_name, driver in drivers.items():
673
nodes = driver.list_nodes()
674
all_nodes[provider_name] = nodes
675
print(f"{provider_name}: {len(nodes)} nodes")
676
```
677
678
### Advanced Deployment with Custom Scripts
679
680
```python
681
from libcloud.compute.deployment import (
682
SSHKeyDeployment, ScriptDeployment, FileDeployment, MultiStepDeployment
683
)
684
685
# Create deployment steps
686
key_deploy = SSHKeyDeployment(public_key_content)
687
688
nginx_config = '''server {
689
listen 80;
690
location / {
691
root /var/www/html;
692
index index.html;
693
}
694
}'''
695
696
config_deploy = FileDeployment("/local/nginx.conf", "/etc/nginx/sites-available/default")
697
698
install_script = ScriptDeployment('''#!/bin/bash
699
apt-get update
700
apt-get install -y nginx
701
systemctl start nginx
702
systemctl enable nginx
703
echo "Hello from Libcloud!" > /var/www/html/index.html
704
systemctl reload nginx
705
''', timeout=300)
706
707
# Combine all steps
708
multi_deploy = MultiStepDeployment([key_deploy, config_deploy, install_script])
709
710
# Deploy node with multi-step deployment
711
deployed_node = driver.deploy_node(
712
name='nginx-server',
713
size=sizes[1],
714
image=images[0],
715
deploy=multi_deploy,
716
ssh_username='ubuntu',
717
timeout=600
718
)
719
720
print(f"Deployed node with nginx: {deployed_node.public_ips[0]}")
721
print(f"Exit status: {install_script.exit_status}")
722
```
723
724
### Volume Management
725
726
```python
727
# Create a storage volume
728
volume = driver.create_volume(
729
size=10, # 10 GB
730
name='data-volume',
731
location=locations[0]
732
)
733
print(f"Created volume: {volume.name} ({volume.id})")
734
735
# Attach volume to a node
736
success = driver.attach_volume(node, volume, device='/dev/sdf')
737
print(f"Volume attached: {success}")
738
739
# Create a snapshot
740
snapshot = driver.create_volume_snapshot(volume, name='backup-snapshot')
741
print(f"Created snapshot: {snapshot.id}")
742
743
# Detach and cleanup
744
driver.detach_volume(volume)
745
driver.destroy_volume_snapshot(snapshot)
746
driver.destroy_volume(volume)
747
```
748
749
## Exception Handling
750
751
```python
752
from libcloud.common.types import InvalidCredsError, LibcloudError
753
from libcloud.compute.types import NodeState
754
755
try:
756
# Attempt operations that may fail
757
driver = cls('invalid_key', 'invalid_secret')
758
nodes = driver.list_nodes()
759
except InvalidCredsError:
760
print("Invalid credentials provided")
761
except LibcloudError as e:
762
print(f"Libcloud error: {e}")
763
764
# Check node state before operations
765
if node.state == NodeState.RUNNING:
766
success = driver.reboot_node(node)
767
elif node.state == NodeState.STOPPED:
768
print("Node is stopped, cannot reboot")
769
```
770
771
## Provider-Specific Features
772
773
Many providers offer additional features through the `ex_*` parameter pattern:
774
775
```python
776
# AWS EC2 specific features
777
node = driver.create_node(
778
name='aws-specific-node',
779
size=size,
780
image=image,
781
ex_security_groups=['web', 'ssh'],
782
ex_subnet_id='subnet-12345',
783
ex_iam_profile='ec2-role',
784
ex_ebs_optimized=True,
785
ex_monitoring=True
786
)
787
788
# Google Compute Engine specific features
789
node = driver.create_node(
790
name='gce-specific-node',
791
size=size,
792
image=image,
793
ex_network='custom-network',
794
ex_tags=['web-server', 'production'],
795
ex_preemptible=True,
796
ex_disk_type='pd-ssd'
797
)
798
```
799
800
Check provider-specific documentation and the driver's `ex_*` methods for additional capabilities.