0
# Resource Path Helpers
1
2
Utility methods for constructing and parsing Google Cloud resource paths for BigQuery Connection resources. These helper methods simplify working with Google Cloud resource names and ensure proper formatting.
3
4
## Capabilities
5
6
### Connection Resource Paths
7
8
Methods for working with BigQuery Connection resource names.
9
10
```python { .api }
11
@staticmethod
12
def connection_path(project: str, location: str, connection: str) -> str:
13
"""
14
Returns a fully-qualified connection string.
15
16
Parameters:
17
- project: Project ID or project number
18
- location: Location/region identifier
19
- connection: Connection identifier
20
21
Returns:
22
str: Connection resource path in format 'projects/{project}/locations/{location}/connections/{connection}'
23
"""
24
25
@staticmethod
26
def parse_connection_path(path: str) -> Dict[str, str]:
27
"""
28
Parses a connection path into its component parts.
29
30
Parameters:
31
- path: Connection resource path
32
33
Returns:
34
Dict[str, str]: Dictionary with keys 'project', 'location', 'connection'
35
36
Raises:
37
ValueError: If the path is not a valid connection path
38
"""
39
```
40
41
**Usage Examples:**
42
43
```python
44
from google.cloud.bigquery_connection import ConnectionServiceClient
45
46
client = ConnectionServiceClient()
47
48
# Construct connection path
49
connection_path = client.connection_path(
50
project="my-project",
51
location="us-central1",
52
connection="analytics-db"
53
)
54
print(connection_path)
55
# Output: projects/my-project/locations/us-central1/connections/analytics-db
56
57
# Parse connection path
58
path_components = client.parse_connection_path(connection_path)
59
print(path_components)
60
# Output: {'project': 'my-project', 'location': 'us-central1', 'connection': 'analytics-db'}
61
62
# Use parsed components
63
project_id = path_components['project']
64
location = path_components['location']
65
connection_id = path_components['connection']
66
67
# Get the connection using parsed values
68
connection = client.get_connection(name=connection_path)
69
```
70
71
### Cluster Resource Paths
72
73
Methods for working with cluster resource names (used with Spark connections).
74
75
```python { .api }
76
@staticmethod
77
def cluster_path(project: str, region: str, cluster: str) -> str:
78
"""
79
Returns a fully-qualified cluster string.
80
81
Parameters:
82
- project: Project ID or project number
83
- region: Region identifier
84
- cluster: Cluster identifier
85
86
Returns:
87
str: Cluster resource path in format 'projects/{project}/regions/{region}/clusters/{cluster}'
88
"""
89
90
@staticmethod
91
def parse_cluster_path(path: str) -> Dict[str, str]:
92
"""
93
Parses a cluster path into its component parts.
94
95
Parameters:
96
- path: Cluster resource path
97
98
Returns:
99
Dict[str, str]: Dictionary with keys 'project', 'region', 'cluster'
100
101
Raises:
102
ValueError: If the path is not a valid cluster path
103
"""
104
```
105
106
**Usage Examples:**
107
108
```python
109
# Construct cluster path for Spark History Server
110
cluster_path = client.cluster_path(
111
project="my-project",
112
region="us-central1",
113
cluster="spark-history-cluster"
114
)
115
print(cluster_path)
116
# Output: projects/my-project/regions/us-central1/clusters/spark-history-cluster
117
118
# Use in Spark connection configuration
119
from google.cloud.bigquery_connection import (
120
Connection,
121
SparkProperties,
122
SparkHistoryServerConfig
123
)
124
125
connection = Connection()
126
connection.spark = SparkProperties()
127
connection.spark.spark_history_server_config = SparkHistoryServerConfig()
128
connection.spark.spark_history_server_config.dataproc_cluster = cluster_path
129
130
# Parse cluster path
131
cluster_components = client.parse_cluster_path(cluster_path)
132
print(f"Cluster: {cluster_components['cluster']} in {cluster_components['region']}")
133
```
134
135
### Service Resource Paths
136
137
Methods for working with service resource names (used with Spark Metastore connections).
138
139
```python { .api }
140
@staticmethod
141
def service_path(project: str, location: str, service: str) -> str:
142
"""
143
Returns a fully-qualified service string.
144
145
Parameters:
146
- project: Project ID or project number
147
- location: Location/region identifier
148
- service: Service identifier
149
150
Returns:
151
str: Service resource path in format 'projects/{project}/locations/{location}/services/{service}'
152
"""
153
154
@staticmethod
155
def parse_service_path(path: str) -> Dict[str, str]:
156
"""
157
Parses a service path into its component parts.
158
159
Parameters:
160
- path: Service resource path
161
162
Returns:
163
Dict[str, str]: Dictionary with keys 'project', 'location', 'service'
164
165
Raises:
166
ValueError: If the path is not a valid service path
167
"""
168
```
169
170
**Usage Examples:**
171
172
```python
173
# Construct service path for Dataproc Metastore
174
service_path = client.service_path(
175
project="my-project",
176
location="us-central1",
177
service="analytics-metastore"
178
)
179
print(service_path)
180
# Output: projects/my-project/locations/us-central1/services/analytics-metastore
181
182
# Use in Spark connection configuration
183
from google.cloud.bigquery_connection import MetastoreServiceConfig
184
185
connection = Connection()
186
connection.spark = SparkProperties()
187
connection.spark.metastore_service_config = MetastoreServiceConfig()
188
connection.spark.metastore_service_config.metastore_service = service_path
189
190
# Parse service path
191
service_components = client.parse_service_path(service_path)
192
print(f"Service: {service_components['service']} in {service_components['location']}")
193
```
194
195
### Common Resource Paths
196
197
Methods for working with common Google Cloud resource paths.
198
199
```python { .api }
200
@staticmethod
201
def common_project_path(project: str) -> str:
202
"""
203
Returns a fully-qualified project string.
204
205
Parameters:
206
- project: Project ID or project number
207
208
Returns:
209
str: Project resource path in format 'projects/{project}'
210
"""
211
212
@staticmethod
213
def parse_common_project_path(path: str) -> Dict[str, str]:
214
"""
215
Parses a project path into its component parts.
216
217
Parameters:
218
- path: Project resource path
219
220
Returns:
221
Dict[str, str]: Dictionary with key 'project'
222
"""
223
224
@staticmethod
225
def common_location_path(project: str, location: str) -> str:
226
"""
227
Returns a fully-qualified location string.
228
229
Parameters:
230
- project: Project ID or project number
231
- location: Location/region identifier
232
233
Returns:
234
str: Location resource path in format 'projects/{project}/locations/{location}'
235
"""
236
237
@staticmethod
238
def parse_common_location_path(path: str) -> Dict[str, str]:
239
"""
240
Parses a location path into its component parts.
241
242
Parameters:
243
- path: Location resource path
244
245
Returns:
246
Dict[str, str]: Dictionary with keys 'project', 'location'
247
"""
248
```
249
250
**Usage Examples:**
251
252
```python
253
# Project path
254
project_path = client.common_project_path("my-project")
255
print(project_path) # Output: projects/my-project
256
257
project_info = client.parse_common_project_path(project_path)
258
print(project_info) # Output: {'project': 'my-project'}
259
260
# Location path (parent for connection operations)
261
location_path = client.common_location_path("my-project", "us-central1")
262
print(location_path) # Output: projects/my-project/locations/us-central1
263
264
location_info = client.parse_common_location_path(location_path)
265
print(location_info) # Output: {'project': 'my-project', 'location': 'us-central1'}
266
267
# Use location path as parent for listing connections
268
connections = client.list_connections(parent=location_path)
269
```
270
271
### Billing Account Paths
272
273
Methods for working with billing account resource paths.
274
275
```python { .api }
276
@staticmethod
277
def common_billing_account_path(billing_account: str) -> str:
278
"""
279
Returns a fully-qualified billing account string.
280
281
Parameters:
282
- billing_account: Billing account ID
283
284
Returns:
285
str: Billing account resource path in format 'billingAccounts/{billing_account}'
286
"""
287
288
@staticmethod
289
def parse_common_billing_account_path(path: str) -> Dict[str, str]:
290
"""
291
Parses a billing account path into its component parts.
292
293
Parameters:
294
- path: Billing account resource path
295
296
Returns:
297
Dict[str, str]: Dictionary with key 'billing_account'
298
"""
299
```
300
301
### Folder and Organization Paths
302
303
Methods for working with resource hierarchy paths.
304
305
```python { .api }
306
@staticmethod
307
def common_folder_path(folder: str) -> str:
308
"""
309
Returns a fully-qualified folder string.
310
311
Parameters:
312
- folder: Folder ID
313
314
Returns:
315
str: Folder resource path in format 'folders/{folder}'
316
"""
317
318
@staticmethod
319
def parse_common_folder_path(path: str) -> Dict[str, str]:
320
"""
321
Parses a folder path into its component parts.
322
323
Parameters:
324
- path: Folder resource path
325
326
Returns:
327
Dict[str, str]: Dictionary with key 'folder'
328
"""
329
330
@staticmethod
331
def common_organization_path(organization: str) -> str:
332
"""
333
Returns a fully-qualified organization string.
334
335
Parameters:
336
- organization: Organization ID
337
338
Returns:
339
str: Organization resource path in format 'organizations/{organization}'
340
"""
341
342
@staticmethod
343
def parse_common_organization_path(path: str) -> Dict[str, str]:
344
"""
345
Parses an organization path into its component parts.
346
347
Parameters:
348
- path: Organization resource path
349
350
Returns:
351
Dict[str, str]: Dictionary with key 'organization'
352
"""
353
```
354
355
## Common Usage Patterns
356
357
### Building Resource Names Dynamically
358
359
```python
360
def create_connection_for_environment(env: str, region: str, conn_type: str):
361
"""Create connections with consistent naming patterns."""
362
client = ConnectionServiceClient()
363
364
# Build paths dynamically
365
project_id = f"analytics-{env}"
366
connection_id = f"{conn_type}-{env}-connection"
367
368
parent = client.common_location_path(project_id, region)
369
connection_name = client.connection_path(project_id, region, connection_id)
370
371
print(f"Parent: {parent}")
372
print(f"Connection: {connection_name}")
373
374
# Create connection
375
connection = Connection()
376
connection.friendly_name = f"{conn_type.title()} {env.title()} Connection"
377
378
result = client.create_connection(
379
parent=parent,
380
connection=connection,
381
connection_id=connection_id
382
)
383
384
return result
385
386
# Usage
387
dev_connection = create_connection_for_environment("dev", "us-central1", "cloudsql")
388
prod_connection = create_connection_for_environment("prod", "us-central1", "cloudsql")
389
```
390
391
### Parsing Resource Names from Configurations
392
393
```python
394
def analyze_connection_config(config: dict):
395
"""Analyze connection configuration and extract resource details."""
396
client = ConnectionServiceClient()
397
398
connections = config.get("connections", [])
399
400
for conn_config in connections:
401
connection_name = conn_config["name"]
402
403
# Parse the connection path
404
try:
405
components = client.parse_connection_path(connection_name)
406
print(f"Connection: {components['connection']}")
407
print(f" Project: {components['project']}")
408
print(f" Location: {components['location']}")
409
410
# Check if connection exists
411
try:
412
connection = client.get_connection(name=connection_name)
413
print(f" Status: Exists ({connection.friendly_name})")
414
except Exception:
415
print(f" Status: Not found")
416
417
except ValueError as e:
418
print(f"Invalid connection path: {connection_name} - {e}")
419
420
# Example configuration
421
config = {
422
"connections": [
423
{"name": "projects/my-project/locations/us-central1/connections/prod-db"},
424
{"name": "projects/my-project/locations/eu-west1/connections/analytics-db"},
425
]
426
}
427
428
analyze_connection_config(config)
429
```
430
431
### Resource Path Validation
432
433
```python
434
def validate_resource_paths(paths: dict) -> dict:
435
"""Validate and normalize resource paths."""
436
client = ConnectionServiceClient()
437
results = {}
438
439
for name, path in paths.items():
440
try:
441
if "/connections/" in path:
442
components = client.parse_connection_path(path)
443
normalized = client.connection_path(**components)
444
results[name] = {"valid": True, "normalized": normalized, "type": "connection"}
445
446
elif "/clusters/" in path:
447
components = client.parse_cluster_path(path)
448
normalized = client.cluster_path(**components)
449
results[name] = {"valid": True, "normalized": normalized, "type": "cluster"}
450
451
elif "/services/" in path and "/locations/" in path:
452
components = client.parse_service_path(path)
453
normalized = client.service_path(**components)
454
results[name] = {"valid": True, "normalized": normalized, "type": "service"}
455
456
else:
457
results[name] = {"valid": False, "error": "Unknown resource type"}
458
459
except ValueError as e:
460
results[name] = {"valid": False, "error": str(e)}
461
462
return results
463
464
# Example usage
465
paths_to_validate = {
466
"main_connection": "projects/my-project/locations/us-central1/connections/analytics",
467
"spark_cluster": "projects/my-project/regions/us-central1/clusters/spark-cluster",
468
"metastore": "projects/my-project/locations/us-central1/services/metastore",
469
"invalid_path": "invalid/path/format"
470
}
471
472
validation_results = validate_resource_paths(paths_to_validate)
473
for name, result in validation_results.items():
474
if result["valid"]:
475
print(f"✓ {name}: {result['type']} - {result['normalized']}")
476
else:
477
print(f"✗ {name}: {result['error']}")
478
```
479
480
## Types
481
482
### Helper Method Return Types
483
484
```python { .api }
485
from typing import Dict
486
487
# All parse methods return string dictionaries
488
PathComponents = Dict[str, str]
489
490
# Example path component structures:
491
ConnectionComponents = Dict[str, str] # Keys: 'project', 'location', 'connection'
492
ClusterComponents = Dict[str, str] # Keys: 'project', 'region', 'cluster'
493
ServiceComponents = Dict[str, str] # Keys: 'project', 'location', 'service'
494
ProjectComponents = Dict[str, str] # Keys: 'project'
495
LocationComponents = Dict[str, str] # Keys: 'project', 'location'
496
BillingComponents = Dict[str, str] # Keys: 'billing_account'
497
FolderComponents = Dict[str, str] # Keys: 'folder'
498
OrganizationComponents = Dict[str, str] # Keys: 'organization'
499
```
500
501
### Resource Name Formats
502
503
```python
504
# Connection resource names
505
CONNECTION_FORMAT = "projects/{project}/locations/{location}/connections/{connection}"
506
507
# Cluster resource names (for Spark)
508
CLUSTER_FORMAT = "projects/{project}/regions/{region}/clusters/{cluster}"
509
510
# Service resource names (for Metastore)
511
SERVICE_FORMAT = "projects/{project}/locations/{location}/services/{service}"
512
513
# Common resource names
514
PROJECT_FORMAT = "projects/{project}"
515
LOCATION_FORMAT = "projects/{project}/locations/{location}"
516
BILLING_ACCOUNT_FORMAT = "billingAccounts/{billing_account}"
517
FOLDER_FORMAT = "folders/{folder}"
518
ORGANIZATION_FORMAT = "organizations/{organization}"
519
```
520
521
### Error Handling
522
523
```python
524
# All parse methods may raise ValueError for invalid paths
525
try:
526
components = client.parse_connection_path(invalid_path)
527
except ValueError as e:
528
print(f"Invalid connection path: {e}")
529
530
# Common validation patterns
531
def is_valid_connection_path(path: str) -> bool:
532
"""Check if a string is a valid connection path."""
533
try:
534
client.parse_connection_path(path)
535
return True
536
except ValueError:
537
return False
538
```