or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

Files

docs

bulk-operations.mddata-catalog.mdentry-metadata.mdindex.mdpolicy-tags.mdtags.mdtaxonomy-serialization.md

data-catalog.mddocs/

0

# Data Catalog Management

1

2

Core catalog operations including search, entry groups, entries, and catalog configuration. This is the primary interface for discovering and managing metadata about data assets across Google Cloud services.

3

4

## Capabilities

5

6

### Catalog Search

7

8

Search for resources across the entire Data Catalog, supporting complex queries with filtering by resource type, properties, and tags.

9

10

```python { .api }

11

def search_catalog(

12

self,

13

request: SearchCatalogRequest = None,

14

*,

15

scope: SearchCatalogRequest.Scope = None,

16

query: str = None,

17

**kwargs

18

) -> SearchCatalogPager:

19

"""

20

Search Data Catalog for resources that match a query.

21

22

Args:

23

request: The request object containing search parameters

24

scope: SearchCatalogRequest.Scope - Required search scope with include_org_ids, include_project_ids, or include_gcp_public_datasets

25

query: str - Required search query using Data Catalog search syntax

26

27

Returns:

28

SearchCatalogPager: Pager for iterating through SearchCatalogResult objects

29

30

Raises:

31

google.api_core.exceptions.InvalidArgument: Invalid search query or scope

32

google.api_core.exceptions.PermissionDenied: Insufficient permissions

33

"""

34

```

35

36

**Usage Example:**

37

38

```python

39

from google.cloud import datacatalog_v1

40

41

client = datacatalog_v1.DataCatalogClient()

42

43

# Search for tables containing "customer" in name or description

44

search_request = datacatalog_v1.SearchCatalogRequest(

45

scope=datacatalog_v1.SearchCatalogRequest.Scope(

46

include_org_ids=["123456789"]

47

),

48

query='type=table AND (name:customer OR description:customer)',

49

page_size=10

50

)

51

52

for result in client.search_catalog(request=search_request):

53

print(f"Found: {result.relative_resource_name}")

54

print(f"Type: {result.search_result_type}")

55

print(f"Display Name: {result.display_name}")

56

```

57

58

### Entry Group Management

59

60

Entry groups provide logical containers for organizing related entries, typically grouping entries from the same data source or project.

61

62

```python { .api }

63

def create_entry_group(

64

self,

65

request: CreateEntryGroupRequest = None,

66

*,

67

parent: str = None,

68

entry_group_id: str = None,

69

entry_group: EntryGroup = None,

70

**kwargs

71

) -> EntryGroup:

72

"""

73

Create an entry group.

74

75

Args:

76

request: The request object

77

parent: str - Required. Format: projects/{project}/locations/{location}

78

entry_group_id: str - Required. ID of the entry group to create

79

entry_group: EntryGroup - Required. The entry group to create

80

81

Returns:

82

EntryGroup: The created entry group

83

84

Raises:

85

google.api_core.exceptions.AlreadyExists: Entry group already exists

86

google.api_core.exceptions.InvalidArgument: Invalid parameters

87

"""

88

89

def get_entry_group(

90

self,

91

request: GetEntryGroupRequest = None,

92

*,

93

name: str = None,

94

read_mask: field_mask_pb2.FieldMask = None,

95

**kwargs

96

) -> EntryGroup:

97

"""

98

Get an entry group.

99

100

Args:

101

request: The request object

102

name: str - Required. Format: projects/{project}/locations/{location}/entryGroups/{entry_group}

103

read_mask: FieldMask - Optional. Fields to return

104

105

Returns:

106

EntryGroup: The requested entry group

107

"""

108

109

def update_entry_group(

110

self,

111

request: UpdateEntryGroupRequest = None,

112

*,

113

entry_group: EntryGroup = None,

114

update_mask: field_mask_pb2.FieldMask = None,

115

**kwargs

116

) -> EntryGroup:

117

"""

118

Update an entry group.

119

120

Args:

121

request: The request object

122

entry_group: EntryGroup - Required. Updated entry group

123

update_mask: FieldMask - Optional. Fields to update

124

125

Returns:

126

EntryGroup: The updated entry group

127

"""

128

129

def delete_entry_group(

130

self,

131

request: DeleteEntryGroupRequest = None,

132

*,

133

name: str = None,

134

force: bool = None,

135

**kwargs

136

) -> None:

137

"""

138

Delete an entry group.

139

140

Args:

141

request: The request object

142

name: str - Required. Format: projects/{project}/locations/{location}/entryGroups/{entry_group}

143

force: bool - Optional. Force delete even if group contains entries

144

"""

145

146

def list_entry_groups(

147

self,

148

request: ListEntryGroupsRequest = None,

149

*,

150

parent: str = None,

151

page_size: int = None,

152

page_token: str = None,

153

**kwargs

154

) -> ListEntryGroupsPager:

155

"""

156

List entry groups.

157

158

Args:

159

request: The request object

160

parent: str - Required. Format: projects/{project}/locations/{location}

161

page_size: int - Optional. Maximum number of results per page

162

page_token: str - Optional. Token for pagination

163

164

Returns:

165

ListEntryGroupsPager: Pager for iterating through EntryGroup objects

166

"""

167

```

168

169

### Entry Management

170

171

Entries represent individual data assets and their metadata, including schema information, business context, and system properties.

172

173

```python { .api }

174

def create_entry(

175

self,

176

request: CreateEntryRequest = None,

177

*,

178

parent: str = None,

179

entry_id: str = None,

180

entry: Entry = None,

181

**kwargs

182

) -> Entry:

183

"""

184

Create an entry.

185

186

Args:

187

request: The request object

188

parent: str - Required. Format: projects/{project}/locations/{location}/entryGroups/{entry_group}

189

entry_id: str - Required. ID of the entry to create

190

entry: Entry - Required. The entry to create

191

192

Returns:

193

Entry: The created entry

194

"""

195

196

def get_entry(

197

self,

198

request: GetEntryRequest = None,

199

*,

200

name: str = None,

201

**kwargs

202

) -> Entry:

203

"""

204

Get an entry.

205

206

Args:

207

request: The request object

208

name: str - Required. Format: projects/{project}/locations/{location}/entryGroups/{entry_group}/entries/{entry}

209

210

Returns:

211

Entry: The requested entry

212

"""

213

214

def update_entry(

215

self,

216

request: UpdateEntryRequest = None,

217

*,

218

entry: Entry = None,

219

update_mask: field_mask_pb2.FieldMask = None,

220

**kwargs

221

) -> Entry:

222

"""

223

Update an entry.

224

225

Args:

226

request: The request object

227

entry: Entry - Required. Updated entry

228

update_mask: FieldMask - Optional. Fields to update

229

230

Returns:

231

Entry: The updated entry

232

"""

233

234

def delete_entry(

235

self,

236

request: DeleteEntryRequest = None,

237

*,

238

name: str = None,

239

**kwargs

240

) -> None:

241

"""

242

Delete an entry.

243

244

Args:

245

request: The request object

246

name: str - Required. Format: projects/{project}/locations/{location}/entryGroups/{entry_group}/entries/{entry}

247

"""

248

249

def list_entries(

250

self,

251

request: ListEntriesRequest = None,

252

*,

253

parent: str = None,

254

page_size: int = None,

255

page_token: str = None,

256

read_mask: field_mask_pb2.FieldMask = None,

257

**kwargs

258

) -> ListEntriesPager:

259

"""

260

List entries.

261

262

Args:

263

request: The request object

264

parent: str - Required. Format: projects/{project}/locations/{location}/entryGroups/{entry_group}

265

page_size: int - Optional. Maximum number of results per page

266

page_token: str - Optional. Token for pagination

267

read_mask: FieldMask - Optional. Fields to return

268

269

Returns:

270

ListEntriesPager: Pager for iterating through Entry objects

271

"""

272

273

def lookup_entry(

274

self,

275

request: LookupEntryRequest = None,

276

*,

277

linked_resource: str = None,

278

sql_resource: str = None,

279

fully_qualified_name: str = None,

280

project: str = None,

281

location: str = None,

282

**kwargs

283

) -> Entry:

284

"""

285

Get an entry by its target resource name.

286

287

Args:

288

request: The request object

289

linked_resource: str - Optional. Full resource name

290

sql_resource: str - Optional. SQL name of the entry

291

fully_qualified_name: str - Optional. Fully qualified name

292

project: str - Optional. Project ID

293

location: str - Optional. Location

294

295

Returns:

296

Entry: The entry for the target resource

297

298

Raises:

299

google.api_core.exceptions.NotFound: Entry not found

300

"""

301

```

302

303

**Usage Example:**

304

305

```python

306

from google.cloud import datacatalog_v1

307

308

client = datacatalog_v1.DataCatalogClient()

309

310

# Create entry group

311

entry_group = datacatalog_v1.EntryGroup(

312

display_name="Customer Data",

313

description="Customer-related data assets"

314

)

315

316

create_group_request = datacatalog_v1.CreateEntryGroupRequest(

317

parent="projects/my-project/locations/us-central1",

318

entry_group_id="customer-data",

319

entry_group=entry_group

320

)

321

322

group = client.create_entry_group(request=create_group_request)

323

324

# Create entry

325

entry = datacatalog_v1.Entry(

326

display_name="Customer Table",

327

description="Main customer information table",

328

type_=datacatalog_v1.EntryType.TABLE,

329

linked_resource="//bigquery.googleapis.com/projects/my-project/datasets/customer/tables/customers"

330

)

331

332

create_entry_request = datacatalog_v1.CreateEntryRequest(

333

parent=group.name,

334

entry_id="customer-table",

335

entry=entry

336

)

337

338

created_entry = client.create_entry(request=create_entry_request)

339

340

# Lookup entry by resource

341

lookup_request = datacatalog_v1.LookupEntryRequest(

342

linked_resource="//bigquery.googleapis.com/projects/my-project/datasets/customer/tables/customers"

343

)

344

found_entry = client.lookup_entry(request=lookup_request)

345

```

346

347

### Configuration Management

348

349

Manage Data Catalog configuration at the organization level, including migration settings and UI experience options.

350

351

```python { .api }

352

def set_config(

353

self,

354

request: SetConfigRequest = None,

355

*,

356

name: str = None,

357

config: OrganizationConfig = None,

358

**kwargs

359

) -> OrganizationConfig:

360

"""

361

Set organization-level Data Catalog configuration.

362

363

Args:

364

request: The request object

365

name: str - Required. Format: organizations/{organization}/locations/{location}/config

366

config: OrganizationConfig - Required. Configuration to set

367

368

Returns:

369

OrganizationConfig: The updated configuration

370

"""

371

372

def retrieve_config(

373

self,

374

request: RetrieveConfigRequest = None,

375

*,

376

name: str = None,

377

**kwargs

378

) -> OrganizationConfig:

379

"""

380

Retrieve organization-level Data Catalog configuration.

381

382

Args:

383

request: The request object

384

name: str - Required. Format: organizations/{organization}/locations/{location}/config

385

386

Returns:

387

OrganizationConfig: The current configuration

388

"""

389

390

def retrieve_effective_config(

391

self,

392

request: RetrieveEffectiveConfigRequest = None,

393

*,

394

name: str = None,

395

**kwargs

396

) -> OrganizationConfig:

397

"""

398

Retrieve effective organization-level Data Catalog configuration.

399

400

Args:

401

request: The request object

402

name: str - Required. Format: projects/{project}/locations/{location}/config

403

404

Returns:

405

OrganizationConfig: The effective configuration

406

"""

407

```

408

409

### IAM Policy Management

410

411

Manage Identity and Access Management (IAM) policies for Data Catalog resources, controlling who can access entries, entry groups, and tag templates.

412

413

```python { .api }

414

def set_iam_policy(

415

self,

416

request: SetIamPolicyRequest = None,

417

*,

418

resource: str = None,

419

**kwargs

420

) -> Policy:

421

"""

422

Set the IAM policy for a resource.

423

424

Args:

425

request: The request object

426

resource: str - Required. Resource name (entry, entry group, or tag template)

427

428

Returns:

429

Policy: The updated IAM policy

430

431

Raises:

432

google.api_core.exceptions.PermissionDenied: Insufficient permissions

433

google.api_core.exceptions.NotFound: Resource not found

434

"""

435

436

def get_iam_policy(

437

self,

438

request: GetIamPolicyRequest = None,

439

*,

440

resource: str = None,

441

**kwargs

442

) -> Policy:

443

"""

444

Get the IAM policy for a resource.

445

446

Args:

447

request: The request object

448

resource: str - Required. Resource name (entry, entry group, or tag template)

449

450

Returns:

451

Policy: The current IAM policy

452

"""

453

454

def test_iam_permissions(

455

self,

456

request: TestIamPermissionsRequest = None,

457

**kwargs

458

) -> TestIamPermissionsResponse:

459

"""

460

Test IAM permissions on a resource.

461

462

Args:

463

request: The request object containing resource and permissions to test

464

465

Returns:

466

TestIamPermissionsResponse: Contains list of permissions the caller has

467

"""

468

```

469

470

**Usage Example:**

471

472

```python

473

from google.cloud import datacatalog_v1

474

from google.iam.v1 import iam_policy_pb2, policy_pb2

475

476

client = datacatalog_v1.DataCatalogClient()

477

478

# Set IAM policy on an entry

479

policy = policy_pb2.Policy(

480

bindings=[

481

policy_pb2.Binding(

482

role="roles/datacatalog.viewer",

483

members=["user:analyst@company.com"]

484

)

485

]

486

)

487

488

set_request = iam_policy_pb2.SetIamPolicyRequest(

489

resource="projects/my-project/locations/us-central1/entryGroups/my-group/entries/my-entry",

490

policy=policy

491

)

492

493

updated_policy = client.set_iam_policy(request=set_request)

494

495

# Test permissions

496

test_request = iam_policy_pb2.TestIamPermissionsRequest(

497

resource="projects/my-project/locations/us-central1/entryGroups/my-group/entries/my-entry",

498

permissions=["datacatalog.entries.get", "datacatalog.entries.update"]

499

)

500

501

permissions_response = client.test_iam_permissions(request=test_request)

502

print(f"Allowed permissions: {permissions_response.permissions}")

503

```

504

505

## Request Types

506

507

```python { .api }

508

class SearchCatalogRequest:

509

scope: SearchCatalogRequest.Scope # Required search scope

510

query: str # Required search query

511

page_size: int # Optional page size

512

page_token: str # Optional pagination token

513

order_by: str # Optional sort order

514

515

class Scope:

516

include_org_ids: Sequence[str] # Organization IDs to include

517

include_project_ids: Sequence[str] # Project IDs to include

518

include_gcp_public_datasets: bool # Include public datasets

519

restricted_locations: Sequence[str] # Restrict to locations

520

starred_only: bool # Only starred entries

521

include_public_tag_templates: bool # Include public tag templates

522

523

class CreateEntryGroupRequest:

524

parent: str # Required parent location

525

entry_group_id: str # Required entry group ID

526

entry_group: EntryGroup # Required entry group

527

528

class CreateEntryRequest:

529

parent: str # Required parent entry group

530

entry_id: str # Required entry ID

531

entry: Entry # Required entry

532

533

class LookupEntryRequest:

534

linked_resource: str # Optional linked resource name

535

sql_resource: str # Optional SQL resource name

536

fully_qualified_name: str # Optional fully qualified name

537

project: str # Optional project ID for FQN lookup

538

location: str # Optional location for FQN lookup

539

540

class SetIamPolicyRequest:

541

resource: str # Required resource name

542

policy: Policy # Required IAM policy to set

543

544

class GetIamPolicyRequest:

545

resource: str # Required resource name

546

options: GetPolicyOptions # Optional policy retrieval options

547

548

class TestIamPermissionsRequest:

549

resource: str # Required resource name

550

permissions: Sequence[str] # Required permissions to test

551

```

552

553

## Response Types

554

555

```python { .api }

556

class SearchCatalogResponse:

557

results: Sequence[SearchCatalogResult] # Search results

558

next_page_token: str # Token for next page

559

unreachable: Sequence[str] # Unreachable locations

560

561

class ListEntryGroupsResponse:

562

entry_groups: Sequence[EntryGroup] # Entry groups

563

next_page_token: str # Token for next page

564

565

class ListEntriesResponse:

566

entries: Sequence[Entry] # Entries

567

next_page_token: str # Token for next page

568

569

class TestIamPermissionsResponse:

570

permissions: Sequence[str] # Permissions the caller has on the resource

571

```