0
# Scheduling and Configuration
1

2
Comprehensive scheduling options and configuration types for defining when and how data transfers execute, including time-based, manual, and event-driven scheduling patterns.
3

4
## Capabilities
5

6
### ScheduleOptions
7

8
Basic scheduling options for data transfers.
9

10
```python { .api }
11
class ScheduleOptions:
12
    """
13
    Options customizing the data transfer schedule.
14
    
15
    Attributes:
16
        disable_auto_scheduling (bool): If set to true, disable automatic scheduling of data transfer runs.
17
        start_time (Timestamp): Specifies time to start scheduling transfer runs.
18
        end_time (Timestamp): Defines time to stop scheduling transfer runs.
19
    """
20
    disable_auto_scheduling: bool
21
    start_time: timestamp_pb2.Timestamp
22
    end_time: timestamp_pb2.Timestamp
23
```
24

25
### ScheduleOptionsV2
26

27
Enhanced scheduling options with more flexibility for data transfers.
28

29
```python { .api }
30
class ScheduleOptionsV2:
31
    """
32
    Options customizing the data transfer schedule (version 2).
33
    
34
    Attributes:
35
        time_based_schedule (TimeBasedSchedule): Time based transfer schedule options.
36
        manual_schedule (ManualSchedule): Manual transfer schedule options.
37
        event_driven_schedule (EventDrivenSchedule): Event driven transfer schedule options.
38
    """
39
    time_based_schedule: TimeBasedSchedule
40
    manual_schedule: ManualSchedule
41
    event_driven_schedule: EventDrivenSchedule
42
```
43

44
### TimeBasedSchedule
45

46
Time-based scheduling configuration for regular, scheduled data transfers.
47

48
```python { .api }
49
class TimeBasedSchedule:
50
    """
51
    Options customizing time based transfer schedule.
52
    
53
    Attributes:
54
        schedule (str): Data transfer schedule in unix-cron format.
55
        start_time (Timestamp): Specifies time to start scheduling transfer runs.
56
        end_time (Timestamp): Defines time to stop scheduling transfer runs.
57
    """
58
    schedule: str
59
    start_time: timestamp_pb2.Timestamp
60
    end_time: timestamp_pb2.Timestamp
61
```
62

63
### ManualSchedule
64

65
Manual scheduling configuration for on-demand data transfers.
66

67
```python { .api }
68
class ManualSchedule:
69
    """
70
    Options customizing manual transfer schedule.
71
    
72
    This message has no fields and is used as a marker for manual scheduling.
73
    """
74
    pass
75
```
76

77
### EventDrivenSchedule
78

79
Event-driven scheduling configuration for data transfers triggered by external events.
80

81
```python { .api }
82
class EventDrivenSchedule:
83
    """
84
    Options customizing event driven transfer schedule.
85
    
86
    Attributes:
87
        pubsub_subscription (str): Pub/Sub subscription name used to receive events.
88
    """
89
    pubsub_subscription: str
90
```
91

92
### EmailPreferences
93

94
Email notification preferences for transfer run events.
95

96
```python { .api }
97
class EmailPreferences:
98
    """
99
    Represents preferences for sending email notifications for transfer run events.
100
    
101
    Attributes:
102
        enable_failure_email (bool): If true, email notifications will be sent on transfer run failures.
103
    """
104
    enable_failure_email: bool
105
```
106

107
### EncryptionConfiguration
108

109
Encryption configuration for data transfers.
110

111
```python { .api }
112
class EncryptionConfiguration:
113
    """
114
    Represents the encryption configuration for data transfers.
115
    
116
    Attributes:
117
        kms_key_name (str): The name of the KMS key used for encryption.
118
    """
119
    kms_key_name: str
120
```
121

122
## Enums and Constants
123

124
### TransferState
125

126
```python { .api }
127
class TransferState(proto.Enum):
128
    """
129
    Represents data transfer run state.
130
    
131
    Values:
132
        TRANSFER_STATE_UNSPECIFIED (0): State placeholder.
133
        PENDING (2): Data transfer is scheduled and waiting to be picked up.
134
        RUNNING (3): Data transfer is in progress.
135
        SUCCEEDED (4): Data transfer completed successfully.
136
        FAILED (5): Data transfer failed.
137
        CANCELLED (6): Data transfer is cancelled.
138
    """
139
    TRANSFER_STATE_UNSPECIFIED = 0
140
    PENDING = 2
141
    RUNNING = 3
142
    SUCCEEDED = 4
143
    FAILED = 5
144
    CANCELLED = 6
145
```
146

147
### TransferType
148

149
```python { .api }
150
class TransferType(proto.Enum):
151
    """
152
    DEPRECATED. Represents data transfer type.
153
    
154
    Values:
155
        TRANSFER_TYPE_UNSPECIFIED (0): Invalid or unknown transfer type placeholder.
156
        BATCH (1): Batch data transfer.
157
        STREAMING (2): Streaming data transfer.
158
    """
159
    TRANSFER_TYPE_UNSPECIFIED = 0
160
    BATCH = 1
161
    STREAMING = 2
162
```
163

164
## Usage Examples
165

166
### Time-Based Scheduling
167

168
```python
169
from google.cloud import bigquery_datatransfer
170
from google.protobuf import timestamp_pb2
171
import datetime
172

173
# Create a time-based schedule
174
start_time = timestamp_pb2.Timestamp()
175
start_time.FromDatetime(datetime.datetime(2024, 1, 1, tzinfo=datetime.timezone.utc))
176

177
end_time = timestamp_pb2.Timestamp() 
178
end_time.FromDatetime(datetime.datetime(2024, 12, 31, tzinfo=datetime.timezone.utc))
179

180
time_based_schedule = bigquery_datatransfer.TimeBasedSchedule(
181
    schedule="every day 08:00",
182
    start_time=start_time,
183
    end_time=end_time
184
)
185

186
schedule_options_v2 = bigquery_datatransfer.ScheduleOptionsV2(
187
    time_based_schedule=time_based_schedule
188
)
189

190
# Use in transfer config
191
transfer_config = {
192
    "display_name": "Daily ETL Transfer",
193
    "data_source_id": "scheduled_query",
194
    "destination_dataset_id": "my_dataset",
195
    "schedule_options_v2": schedule_options_v2,
196
    # ... other config options
197
}
198
```
199

200
### Manual Scheduling
201

202
```python
203
from google.cloud import bigquery_datatransfer
204

205
# Create manual schedule (no automatic runs)
206
manual_schedule = bigquery_datatransfer.ManualSchedule()
207

208
schedule_options_v2 = bigquery_datatransfer.ScheduleOptionsV2(
209
    manual_schedule=manual_schedule
210
)
211

212
# Use in transfer config for manual-only runs
213
transfer_config = {
214
    "display_name": "Manual ETL Transfer",
215
    "data_source_id": "scheduled_query",
216
    "destination_dataset_id": "my_dataset",
217
    "schedule_options_v2": schedule_options_v2,
218
    # ... other config options
219
}
220
```
221

222
### Event-Driven Scheduling
223

224
```python
225
from google.cloud import bigquery_datatransfer
226

227
# Create event-driven schedule
228
event_driven_schedule = bigquery_datatransfer.EventDrivenSchedule(
229
    pubsub_subscription=f"projects/{project_id}/subscriptions/data-transfer-events"
230
)
231

232
schedule_options_v2 = bigquery_datatransfer.ScheduleOptionsV2(
233
    event_driven_schedule=event_driven_schedule
234
)
235

236
# Use in transfer config for event-triggered runs
237
transfer_config = {
238
    "display_name": "Event-Driven ETL Transfer",
239
    "data_source_id": "scheduled_query",
240
    "destination_dataset_id": "my_dataset",
241
    "schedule_options_v2": schedule_options_v2,
242
    # ... other config options
243
}
244
```
245

246
### Basic Schedule Options (Legacy)
247

248
```python
249
from google.cloud import bigquery_datatransfer
250
from google.protobuf import timestamp_pb2
251
import datetime
252

253
# Create basic schedule options
254
start_time = timestamp_pb2.Timestamp()
255
start_time.FromDatetime(datetime.datetime(2024, 1, 1, tzinfo=datetime.timezone.utc))
256

257
end_time = timestamp_pb2.Timestamp()
258
end_time.FromDatetime(datetime.datetime(2024, 12, 31, tzinfo=datetime.timezone.utc))
259

260
schedule_options = bigquery_datatransfer.ScheduleOptions(
261
    disable_auto_scheduling=False,
262
    start_time=start_time,
263
    end_time=end_time
264
)
265

266
# Use with regular schedule string
267
transfer_config = {
268
    "display_name": "Scheduled ETL Transfer",
269
    "data_source_id": "scheduled_query",
270
    "destination_dataset_id": "my_dataset",
271
    "schedule": "every day 08:00",
272
    "schedule_options": schedule_options,
273
    # ... other config options
274
}
275
```
276

277
### Email Preferences Configuration
278

279
```python
280
from google.cloud import bigquery_datatransfer
281

282
# Configure email notifications
283
email_preferences = bigquery_datatransfer.EmailPreferences(
284
    enable_failure_email=True
285
)
286

287
# Use in transfer config
288
transfer_config = {
289
    "display_name": "Monitored ETL Transfer",
290
    "data_source_id": "scheduled_query",
291
    "destination_dataset_id": "my_dataset",
292
    "schedule": "every day 08:00",
293
    "email_preferences": email_preferences,
294
    # ... other config options
295
}
296
```
297

298
### Encryption Configuration
299

300
```python
301
from google.cloud import bigquery_datatransfer
302

303
# Configure encryption with customer-managed key
304
encryption_config = bigquery_datatransfer.EncryptionConfiguration(
305
    kms_key_name=f"projects/{project_id}/locations/{location}/keyRings/{key_ring}/cryptoKeys/{key}"
306
)
307

308
# Use in transfer config
309
transfer_config = {
310
    "display_name": "Encrypted ETL Transfer",
311
    "data_source_id": "scheduled_query",
312
    "destination_dataset_id": "my_dataset",
313
    "schedule": "every day 08:00",
314
    "encryption_configuration": encryption_config,
315
    # ... other config options
316
}
317
```
318

319
### Working with Transfer States
320

321
```python
322
from google.cloud import bigquery_datatransfer
323
from google.cloud.bigquery_datatransfer_v1 import TransferState
324

325
client = bigquery_datatransfer.DataTransferServiceClient()
326

327
# List only failed runs
328
parent = f"projects/{project_id}/locations/{location}/transferConfigs/{config_id}"
329
response = client.list_transfer_runs(
330
    parent=parent,
331
    states=[TransferState.FAILED]
332
)
333

334
print("Failed transfer runs:")
335
for run in response:
336
    print(f"  Run ID: {run.name}")
337
    print(f"  State: {run.state}")
338
    print(f"  Error: {run.error_status.message if run.error_status else 'No error details'}")
339

340
# Check run state
341
def get_state_description(state):
342
    if state == TransferState.PENDING:
343
        return "Transfer is scheduled and waiting"
344
    elif state == TransferState.RUNNING:
345
        return "Transfer is currently running"
346
    elif state == TransferState.SUCCEEDED:
347
        return "Transfer completed successfully"
348
    elif state == TransferState.FAILED:
349
        return "Transfer failed"
350
    elif state == TransferState.CANCELLED:
351
        return "Transfer was cancelled"
352
    else:
353
        return "Unknown state"
354

355
# Example usage
356
run = client.get_transfer_run(name=run_name)
357
print(f"Transfer state: {get_state_description(run.state)}")
358
```
359

360
## Schedule Format Examples
361

362
Common schedule formats for time-based scheduling:
363

364
```python
365
# Daily schedules
366
"every day 08:00"                    # Every day at 8 AM
367
"every 2 days 14:30"                 # Every 2 days at 2:30 PM
368

369
# Weekly schedules  
370
"every monday 09:00"                 # Every Monday at 9 AM
371
"every sunday 23:00"                 # Every Sunday at 11 PM
372

373
# Monthly schedules
374
"1st,15th of month 10:00"           # 1st and 15th of each month at 10 AM
375
"last day of month 18:00"           # Last day of each month at 6 PM
376

377
# Hourly schedules
378
"every 4 hours"                      # Every 4 hours
379
"every hour"                         # Every hour
380

381
# Custom cron expressions
382
"0 */6 * * *"                       # Every 6 hours (cron format)
383
"0 0 * * 1"                         # Every Monday at midnight (cron format)
384
```