0
# Data Catalog Management
1
2
Core catalog operations including search, entry groups, entries, and catalog configuration. This is the primary interface for discovering and managing metadata about data assets across Google Cloud services.
3
4
## Capabilities
5
6
### Catalog Search
7
8
Search for resources across the entire Data Catalog, supporting complex queries with filtering by resource type, properties, and tags.
9
10
```python { .api }
11
def search_catalog(
12
self,
13
request: SearchCatalogRequest = None,
14
*,
15
scope: SearchCatalogRequest.Scope = None,
16
query: str = None,
17
**kwargs
18
) -> SearchCatalogPager:
19
"""
20
Search Data Catalog for resources that match a query.
21
22
Args:
23
request: The request object containing search parameters
24
scope: SearchCatalogRequest.Scope - Required search scope with include_org_ids, include_project_ids, or include_gcp_public_datasets
25
query: str - Required search query using Data Catalog search syntax
26
27
Returns:
28
SearchCatalogPager: Pager for iterating through SearchCatalogResult objects
29
30
Raises:
31
google.api_core.exceptions.InvalidArgument: Invalid search query or scope
32
google.api_core.exceptions.PermissionDenied: Insufficient permissions
33
"""
34
```
35
36
**Usage Example:**
37
38
```python
39
from google.cloud import datacatalog_v1
40
41
client = datacatalog_v1.DataCatalogClient()
42
43
# Search for tables containing "customer" in name or description
44
search_request = datacatalog_v1.SearchCatalogRequest(
45
scope=datacatalog_v1.SearchCatalogRequest.Scope(
46
include_org_ids=["123456789"]
47
),
48
query='type=table AND (name:customer OR description:customer)',
49
page_size=10
50
)
51
52
for result in client.search_catalog(request=search_request):
53
print(f"Found: {result.relative_resource_name}")
54
print(f"Type: {result.search_result_type}")
55
print(f"Display Name: {result.display_name}")
56
```
57
58
### Entry Group Management
59
60
Entry groups provide logical containers for organizing related entries, typically grouping entries from the same data source or project.
61
62
```python { .api }
63
def create_entry_group(
64
self,
65
request: CreateEntryGroupRequest = None,
66
*,
67
parent: str = None,
68
entry_group_id: str = None,
69
entry_group: EntryGroup = None,
70
**kwargs
71
) -> EntryGroup:
72
"""
73
Create an entry group.
74
75
Args:
76
request: The request object
77
parent: str - Required. Format: projects/{project}/locations/{location}
78
entry_group_id: str - Required. ID of the entry group to create
79
entry_group: EntryGroup - Required. The entry group to create
80
81
Returns:
82
EntryGroup: The created entry group
83
84
Raises:
85
google.api_core.exceptions.AlreadyExists: Entry group already exists
86
google.api_core.exceptions.InvalidArgument: Invalid parameters
87
"""
88
89
def get_entry_group(
90
self,
91
request: GetEntryGroupRequest = None,
92
*,
93
name: str = None,
94
read_mask: field_mask_pb2.FieldMask = None,
95
**kwargs
96
) -> EntryGroup:
97
"""
98
Get an entry group.
99
100
Args:
101
request: The request object
102
name: str - Required. Format: projects/{project}/locations/{location}/entryGroups/{entry_group}
103
read_mask: FieldMask - Optional. Fields to return
104
105
Returns:
106
EntryGroup: The requested entry group
107
"""
108
109
def update_entry_group(
110
self,
111
request: UpdateEntryGroupRequest = None,
112
*,
113
entry_group: EntryGroup = None,
114
update_mask: field_mask_pb2.FieldMask = None,
115
**kwargs
116
) -> EntryGroup:
117
"""
118
Update an entry group.
119
120
Args:
121
request: The request object
122
entry_group: EntryGroup - Required. Updated entry group
123
update_mask: FieldMask - Optional. Fields to update
124
125
Returns:
126
EntryGroup: The updated entry group
127
"""
128
129
def delete_entry_group(
130
self,
131
request: DeleteEntryGroupRequest = None,
132
*,
133
name: str = None,
134
force: bool = None,
135
**kwargs
136
) -> None:
137
"""
138
Delete an entry group.
139
140
Args:
141
request: The request object
142
name: str - Required. Format: projects/{project}/locations/{location}/entryGroups/{entry_group}
143
force: bool - Optional. Force delete even if group contains entries
144
"""
145
146
def list_entry_groups(
147
self,
148
request: ListEntryGroupsRequest = None,
149
*,
150
parent: str = None,
151
page_size: int = None,
152
page_token: str = None,
153
**kwargs
154
) -> ListEntryGroupsPager:
155
"""
156
List entry groups.
157
158
Args:
159
request: The request object
160
parent: str - Required. Format: projects/{project}/locations/{location}
161
page_size: int - Optional. Maximum number of results per page
162
page_token: str - Optional. Token for pagination
163
164
Returns:
165
ListEntryGroupsPager: Pager for iterating through EntryGroup objects
166
"""
167
```
168
169
### Entry Management
170
171
Entries represent individual data assets and their metadata, including schema information, business context, and system properties.
172
173
```python { .api }
174
def create_entry(
175
self,
176
request: CreateEntryRequest = None,
177
*,
178
parent: str = None,
179
entry_id: str = None,
180
entry: Entry = None,
181
**kwargs
182
) -> Entry:
183
"""
184
Create an entry.
185
186
Args:
187
request: The request object
188
parent: str - Required. Format: projects/{project}/locations/{location}/entryGroups/{entry_group}
189
entry_id: str - Required. ID of the entry to create
190
entry: Entry - Required. The entry to create
191
192
Returns:
193
Entry: The created entry
194
"""
195
196
def get_entry(
197
self,
198
request: GetEntryRequest = None,
199
*,
200
name: str = None,
201
**kwargs
202
) -> Entry:
203
"""
204
Get an entry.
205
206
Args:
207
request: The request object
208
name: str - Required. Format: projects/{project}/locations/{location}/entryGroups/{entry_group}/entries/{entry}
209
210
Returns:
211
Entry: The requested entry
212
"""
213
214
def update_entry(
215
self,
216
request: UpdateEntryRequest = None,
217
*,
218
entry: Entry = None,
219
update_mask: field_mask_pb2.FieldMask = None,
220
**kwargs
221
) -> Entry:
222
"""
223
Update an entry.
224
225
Args:
226
request: The request object
227
entry: Entry - Required. Updated entry
228
update_mask: FieldMask - Optional. Fields to update
229
230
Returns:
231
Entry: The updated entry
232
"""
233
234
def delete_entry(
235
self,
236
request: DeleteEntryRequest = None,
237
*,
238
name: str = None,
239
**kwargs
240
) -> None:
241
"""
242
Delete an entry.
243
244
Args:
245
request: The request object
246
name: str - Required. Format: projects/{project}/locations/{location}/entryGroups/{entry_group}/entries/{entry}
247
"""
248
249
def list_entries(
250
self,
251
request: ListEntriesRequest = None,
252
*,
253
parent: str = None,
254
page_size: int = None,
255
page_token: str = None,
256
read_mask: field_mask_pb2.FieldMask = None,
257
**kwargs
258
) -> ListEntriesPager:
259
"""
260
List entries.
261
262
Args:
263
request: The request object
264
parent: str - Required. Format: projects/{project}/locations/{location}/entryGroups/{entry_group}
265
page_size: int - Optional. Maximum number of results per page
266
page_token: str - Optional. Token for pagination
267
read_mask: FieldMask - Optional. Fields to return
268
269
Returns:
270
ListEntriesPager: Pager for iterating through Entry objects
271
"""
272
273
def lookup_entry(
274
self,
275
request: LookupEntryRequest = None,
276
*,
277
linked_resource: str = None,
278
sql_resource: str = None,
279
fully_qualified_name: str = None,
280
project: str = None,
281
location: str = None,
282
**kwargs
283
) -> Entry:
284
"""
285
Get an entry by its target resource name.
286
287
Args:
288
request: The request object
289
linked_resource: str - Optional. Full resource name
290
sql_resource: str - Optional. SQL name of the entry
291
fully_qualified_name: str - Optional. Fully qualified name
292
project: str - Optional. Project ID
293
location: str - Optional. Location
294
295
Returns:
296
Entry: The entry for the target resource
297
298
Raises:
299
google.api_core.exceptions.NotFound: Entry not found
300
"""
301
```
302
303
**Usage Example:**
304
305
```python
306
from google.cloud import datacatalog_v1
307
308
client = datacatalog_v1.DataCatalogClient()
309
310
# Create entry group
311
entry_group = datacatalog_v1.EntryGroup(
312
display_name="Customer Data",
313
description="Customer-related data assets"
314
)
315
316
create_group_request = datacatalog_v1.CreateEntryGroupRequest(
317
parent="projects/my-project/locations/us-central1",
318
entry_group_id="customer-data",
319
entry_group=entry_group
320
)
321
322
group = client.create_entry_group(request=create_group_request)
323
324
# Create entry
325
entry = datacatalog_v1.Entry(
326
display_name="Customer Table",
327
description="Main customer information table",
328
type_=datacatalog_v1.EntryType.TABLE,
329
linked_resource="//bigquery.googleapis.com/projects/my-project/datasets/customer/tables/customers"
330
)
331
332
create_entry_request = datacatalog_v1.CreateEntryRequest(
333
parent=group.name,
334
entry_id="customer-table",
335
entry=entry
336
)
337
338
created_entry = client.create_entry(request=create_entry_request)
339
340
# Lookup entry by resource
341
lookup_request = datacatalog_v1.LookupEntryRequest(
342
linked_resource="//bigquery.googleapis.com/projects/my-project/datasets/customer/tables/customers"
343
)
344
found_entry = client.lookup_entry(request=lookup_request)
345
```
346
347
### Configuration Management
348
349
Manage Data Catalog configuration at the organization level, including migration settings and UI experience options.
350
351
```python { .api }
352
def set_config(
353
self,
354
request: SetConfigRequest = None,
355
*,
356
name: str = None,
357
config: OrganizationConfig = None,
358
**kwargs
359
) -> OrganizationConfig:
360
"""
361
Set organization-level Data Catalog configuration.
362
363
Args:
364
request: The request object
365
name: str - Required. Format: organizations/{organization}/locations/{location}/config
366
config: OrganizationConfig - Required. Configuration to set
367
368
Returns:
369
OrganizationConfig: The updated configuration
370
"""
371
372
def retrieve_config(
373
self,
374
request: RetrieveConfigRequest = None,
375
*,
376
name: str = None,
377
**kwargs
378
) -> OrganizationConfig:
379
"""
380
Retrieve organization-level Data Catalog configuration.
381
382
Args:
383
request: The request object
384
name: str - Required. Format: organizations/{organization}/locations/{location}/config
385
386
Returns:
387
OrganizationConfig: The current configuration
388
"""
389
390
def retrieve_effective_config(
391
self,
392
request: RetrieveEffectiveConfigRequest = None,
393
*,
394
name: str = None,
395
**kwargs
396
) -> OrganizationConfig:
397
"""
398
Retrieve effective organization-level Data Catalog configuration.
399
400
Args:
401
request: The request object
402
name: str - Required. Format: projects/{project}/locations/{location}/config
403
404
Returns:
405
OrganizationConfig: The effective configuration
406
"""
407
```
408
409
### IAM Policy Management
410
411
Manage Identity and Access Management (IAM) policies for Data Catalog resources, controlling who can access entries, entry groups, and tag templates.
412
413
```python { .api }
414
def set_iam_policy(
415
self,
416
request: SetIamPolicyRequest = None,
417
*,
418
resource: str = None,
419
**kwargs
420
) -> Policy:
421
"""
422
Set the IAM policy for a resource.
423
424
Args:
425
request: The request object
426
resource: str - Required. Resource name (entry, entry group, or tag template)
427
428
Returns:
429
Policy: The updated IAM policy
430
431
Raises:
432
google.api_core.exceptions.PermissionDenied: Insufficient permissions
433
google.api_core.exceptions.NotFound: Resource not found
434
"""
435
436
def get_iam_policy(
437
self,
438
request: GetIamPolicyRequest = None,
439
*,
440
resource: str = None,
441
**kwargs
442
) -> Policy:
443
"""
444
Get the IAM policy for a resource.
445
446
Args:
447
request: The request object
448
resource: str - Required. Resource name (entry, entry group, or tag template)
449
450
Returns:
451
Policy: The current IAM policy
452
"""
453
454
def test_iam_permissions(
455
self,
456
request: TestIamPermissionsRequest = None,
457
**kwargs
458
) -> TestIamPermissionsResponse:
459
"""
460
Test IAM permissions on a resource.
461
462
Args:
463
request: The request object containing resource and permissions to test
464
465
Returns:
466
TestIamPermissionsResponse: Contains list of permissions the caller has
467
"""
468
```
469
470
**Usage Example:**
471
472
```python
473
from google.cloud import datacatalog_v1
474
from google.iam.v1 import iam_policy_pb2, policy_pb2
475
476
client = datacatalog_v1.DataCatalogClient()
477
478
# Set IAM policy on an entry
479
policy = policy_pb2.Policy(
480
bindings=[
481
policy_pb2.Binding(
482
role="roles/datacatalog.viewer",
483
members=["user:analyst@company.com"]
484
)
485
]
486
)
487
488
set_request = iam_policy_pb2.SetIamPolicyRequest(
489
resource="projects/my-project/locations/us-central1/entryGroups/my-group/entries/my-entry",
490
policy=policy
491
)
492
493
updated_policy = client.set_iam_policy(request=set_request)
494
495
# Test permissions
496
test_request = iam_policy_pb2.TestIamPermissionsRequest(
497
resource="projects/my-project/locations/us-central1/entryGroups/my-group/entries/my-entry",
498
permissions=["datacatalog.entries.get", "datacatalog.entries.update"]
499
)
500
501
permissions_response = client.test_iam_permissions(request=test_request)
502
print(f"Allowed permissions: {permissions_response.permissions}")
503
```
504
505
## Request Types
506
507
```python { .api }
508
class SearchCatalogRequest:
509
scope: SearchCatalogRequest.Scope # Required search scope
510
query: str # Required search query
511
page_size: int # Optional page size
512
page_token: str # Optional pagination token
513
order_by: str # Optional sort order
514
515
class Scope:
516
include_org_ids: Sequence[str] # Organization IDs to include
517
include_project_ids: Sequence[str] # Project IDs to include
518
include_gcp_public_datasets: bool # Include public datasets
519
restricted_locations: Sequence[str] # Restrict to locations
520
starred_only: bool # Only starred entries
521
include_public_tag_templates: bool # Include public tag templates
522
523
class CreateEntryGroupRequest:
524
parent: str # Required parent location
525
entry_group_id: str # Required entry group ID
526
entry_group: EntryGroup # Required entry group
527
528
class CreateEntryRequest:
529
parent: str # Required parent entry group
530
entry_id: str # Required entry ID
531
entry: Entry # Required entry
532
533
class LookupEntryRequest:
534
linked_resource: str # Optional linked resource name
535
sql_resource: str # Optional SQL resource name
536
fully_qualified_name: str # Optional fully qualified name
537
project: str # Optional project ID for FQN lookup
538
location: str # Optional location for FQN lookup
539
540
class SetIamPolicyRequest:
541
resource: str # Required resource name
542
policy: Policy # Required IAM policy to set
543
544
class GetIamPolicyRequest:
545
resource: str # Required resource name
546
options: GetPolicyOptions # Optional policy retrieval options
547
548
class TestIamPermissionsRequest:
549
resource: str # Required resource name
550
permissions: Sequence[str] # Required permissions to test
551
```
552
553
## Response Types
554
555
```python { .api }
556
class SearchCatalogResponse:
557
results: Sequence[SearchCatalogResult] # Search results
558
next_page_token: str # Token for next page
559
unreachable: Sequence[str] # Unreachable locations
560
561
class ListEntryGroupsResponse:
562
entry_groups: Sequence[EntryGroup] # Entry groups
563
next_page_token: str # Token for next page
564
565
class ListEntriesResponse:
566
entries: Sequence[Entry] # Entries
567
next_page_token: str # Token for next page
568
569
class TestIamPermissionsResponse:
570
permissions: Sequence[str] # Permissions the caller has on the resource
571
```