0
# Data Models and Configuration
1
2
Core data structures, retry policies, message encoding options, service configuration models, and error handling components for azure-storage-queue operations.
3
4
## Capabilities
5
6
### Message Encoding Policies
7
8
Custom policies for encoding and decoding message content to handle different data types and serialization requirements.
9
10
```python { .api }
11
class TextBase64EncodePolicy:
12
"""Base64 encoding for text message content."""
13
14
def encode(self, content: str) -> str:
15
"""
16
Encode text content to base64.
17
18
Parameters:
19
- content: Text string to encode
20
21
Returns:
22
Base64 encoded string
23
"""
24
25
class TextBase64DecodePolicy:
26
"""Base64 decoding for text message content."""
27
28
def decode(self, content: str, response) -> str:
29
"""
30
Decode base64 content to text.
31
32
Parameters:
33
- content: Base64 encoded string
34
- response: HTTP response object
35
36
Returns:
37
Decoded text string
38
"""
39
40
class BinaryBase64EncodePolicy:
41
"""Base64 encoding for binary message content."""
42
43
def encode(self, content: bytes) -> str:
44
"""
45
Encode binary content to base64.
46
47
Parameters:
48
- content: Binary data to encode
49
50
Returns:
51
Base64 encoded string
52
"""
53
54
class BinaryBase64DecodePolicy:
55
"""Base64 decoding for binary message content."""
56
57
def decode(self, content: str, response) -> bytes:
58
"""
59
Decode base64 content to binary.
60
61
Parameters:
62
- content: Base64 encoded string
63
- response: HTTP response object
64
65
Returns:
66
Decoded binary data
67
"""
68
```
69
70
### Retry Policies
71
72
Configurable retry mechanisms for handling transient failures and network issues.
73
74
```python { .api }
75
class ExponentialRetry:
76
"""Exponential backoff retry policy with jitter."""
77
78
def __init__(
79
self,
80
initial_backoff: int = 15,
81
increment_base: int = 3,
82
retry_total: int = 3,
83
retry_to_secondary: bool = False,
84
random_jitter_range: int = 3,
85
**kwargs
86
):
87
"""
88
Create exponential retry policy.
89
90
Parameters:
91
- initial_backoff: Initial backoff interval in seconds
92
- increment_base: Multiplier for exponential backoff
93
- retry_total: Maximum number of retry attempts
94
- retry_to_secondary: Whether to retry to secondary endpoint
95
- random_jitter_range: Random jitter range in seconds
96
"""
97
98
initial_backoff: int
99
increment_base: int
100
random_jitter_range: int
101
retry_total: int
102
retry_to_secondary: bool
103
104
class LinearRetry:
105
"""Linear backoff retry policy with jitter."""
106
107
def __init__(
108
self,
109
backoff: int = 15,
110
retry_total: int = 3,
111
retry_to_secondary: bool = False,
112
random_jitter_range: int = 3,
113
**kwargs
114
):
115
"""
116
Create linear retry policy.
117
118
Parameters:
119
- backoff: Fixed backoff interval in seconds
120
- retry_total: Maximum number of retry attempts
121
- retry_to_secondary: Whether to retry to secondary endpoint
122
- random_jitter_range: Random jitter range in seconds
123
"""
124
125
backoff: int
126
random_jitter_range: int
127
retry_total: int
128
retry_to_secondary: bool
129
```
130
131
### Service Configuration Models
132
133
Models for configuring storage service properties including analytics, metrics, and CORS policies.
134
135
```python { .api }
136
class QueueAnalyticsLogging:
137
"""Azure Storage Analytics logging configuration."""
138
139
version: str # Analytics version (default: "1.0")
140
delete: bool # Log delete requests
141
read: bool # Log read requests
142
write: bool # Log write requests
143
retention_policy: RetentionPolicy # Log retention settings
144
145
@classmethod
146
def _from_generated(cls, generated) -> 'QueueAnalyticsLogging': ...
147
148
class Metrics:
149
"""Storage Analytics metrics configuration."""
150
151
version: str # Analytics version (default: "1.0")
152
enabled: bool # Whether metrics collection is enabled
153
include_apis: Optional[bool] # Include API operation statistics
154
retention_policy: RetentionPolicy # Metrics retention settings
155
156
@classmethod
157
def _from_generated(cls, generated) -> 'Metrics': ...
158
159
class RetentionPolicy:
160
"""Data retention policy for logs and metrics."""
161
162
enabled: bool # Whether retention policy is enabled
163
days: Optional[int] # Number of days to retain data (1-365)
164
165
@classmethod
166
def _from_generated(cls, generated) -> 'RetentionPolicy': ...
167
168
class CorsRule:
169
"""Cross-Origin Resource Sharing (CORS) rule."""
170
171
allowed_origins: str # Comma-separated allowed origin domains
172
allowed_methods: str # Comma-separated allowed HTTP methods
173
max_age_in_seconds: int # Preflight response cache duration
174
exposed_headers: str # Comma-separated response headers to expose
175
allowed_headers: str # Comma-separated allowed request headers
176
177
@staticmethod
178
def _to_generated(rules: Optional[List['CorsRule']]) -> Optional[List]: ...
179
180
@classmethod
181
def _from_generated(cls, generated) -> 'CorsRule': ...
182
```
183
184
### Location and Error Models
185
186
Location routing and comprehensive error code definitions for storage operations.
187
188
```python { .api }
189
class LocationMode:
190
"""Location mode for geo-redundant storage requests."""
191
192
PRIMARY: str = "primary" # Route requests to primary location
193
SECONDARY: str = "secondary" # Route requests to secondary location
194
195
class StorageErrorCode:
196
"""Error codes returned by Azure Storage service."""
197
198
# Queue-specific error codes
199
INVALID_MARKER: str # Invalid marker value in list operation
200
MESSAGE_NOT_FOUND: str # Specified message does not exist
201
MESSAGE_TOO_LARGE: str # Message exceeds maximum size limit
202
POP_RECEIPT_MISMATCH: str # Pop receipt does not match message
203
QUEUE_ALREADY_EXISTS: str # Queue already exists
204
QUEUE_BEING_DELETED: str # Queue is currently being deleted
205
QUEUE_DISABLED: str # Queue is disabled
206
QUEUE_NOT_EMPTY: str # Queue contains messages
207
QUEUE_NOT_FOUND: str # Specified queue does not exist
208
209
# Additional common error codes
210
INVALID_AUTHENTICATION_INFO: str # Authentication information is invalid
211
INVALID_RESOURCE_NAME: str # Resource name is invalid
212
REQUEST_BODY_TOO_LARGE: str # Request body exceeds size limit
213
INVALID_HEADER_VALUE: str # Header value is invalid
214
MISSING_REQUIRED_HEADER: str # Required header is missing
215
UNSUPPORTED_HEADER: str # Header is not supported
216
INVALID_QUERY_PARAMETER_VALUE: str # Query parameter value is invalid
217
OUT_OF_RANGE_QUERY_PARAMETER_VALUE: str # Query parameter value is out of range
218
INVALID_URI: str # Request URI is invalid
219
INVALID_HTTP_VERB: str # HTTP verb is not supported
220
EMPTY_METADATA_KEY: str # Metadata key is empty
221
REQUEST_URL_FAILED_TO_PARSE: str # Request URL could not be parsed
222
INVALID_XML_DOCUMENT: str # XML document is invalid
223
INVALID_XML_NODE_VALUE: str # XML node value is invalid
224
MISSING_REQUIRED_XML_NODE: str # Required XML node is missing
225
UNSUPPORTED_XML_NODE: str # XML node is not supported
226
```
227
228
## Usage Examples
229
230
### Custom Message Encoding
231
232
```python
233
from azure.storage.queue import QueueClient, BinaryBase64EncodePolicy, BinaryBase64DecodePolicy
234
import json
235
import pickle
236
237
# Create client with custom encoding policies
238
queue_client = QueueClient.from_connection_string(
239
conn_str,
240
"myqueue",
241
message_encode_policy=BinaryBase64EncodePolicy(),
242
message_decode_policy=BinaryBase64DecodePolicy()
243
)
244
245
# Send JSON data as binary
246
data = {"user_id": 12345, "action": "purchase", "amount": 99.99}
247
json_bytes = json.dumps(data).encode('utf-8')
248
queue_client.send_message(json_bytes)
249
250
# Send pickled Python objects
251
python_obj = {"complex": [1, 2, 3], "nested": {"key": "value"}}
252
pickled_data = pickle.dumps(python_obj)
253
queue_client.send_message(pickled_data)
254
255
# Receive and decode
256
message = queue_client.receive_message()
257
if message:
258
# message.content will be bytes due to BinaryBase64DecodePolicy
259
decoded_data = json.loads(message.content.decode('utf-8'))
260
print(f"Received: {decoded_data}")
261
```
262
263
### Retry Policy Configuration
264
265
```python
266
from azure.storage.queue import QueueServiceClient, ExponentialRetry, LinearRetry
267
268
# Configure exponential backoff
269
exponential_retry = ExponentialRetry(
270
initial_backoff=10, # Start with 10 second delay
271
increment_base=2, # Double the delay each retry
272
retry_total=5, # Maximum 5 retry attempts
273
retry_to_secondary=True, # Try secondary endpoint if available
274
random_jitter_range=5 # Add 0-5 seconds random jitter
275
)
276
277
service_client = QueueServiceClient.from_connection_string(
278
conn_str,
279
retry_policy=exponential_retry
280
)
281
282
# Configure linear backoff
283
linear_retry = LinearRetry(
284
backoff=30, # Fixed 30 second delay between retries
285
retry_total=3, # Maximum 3 retry attempts
286
random_jitter_range=10 # Add 0-10 seconds random jitter
287
)
288
289
queue_client = service_client.get_queue_client(
290
"myqueue",
291
retry_policy=linear_retry
292
)
293
```
294
295
### Service Properties Configuration
296
297
```python
298
from azure.storage.queue import (
299
QueueServiceClient, QueueAnalyticsLogging, Metrics,
300
RetentionPolicy, CorsRule
301
)
302
303
service_client = QueueServiceClient.from_connection_string(conn_str)
304
305
# Configure comprehensive logging
306
logging_config = QueueAnalyticsLogging(
307
version="1.0",
308
delete=True, # Log delete operations
309
read=True, # Log read operations
310
write=True, # Log write operations
311
retention_policy=RetentionPolicy(enabled=True, days=30)
312
)
313
314
# Configure detailed metrics
315
metrics_config = Metrics(
316
version="1.0",
317
enabled=True,
318
include_apis=True, # Include per-API statistics
319
retention_policy=RetentionPolicy(enabled=True, days=90)
320
)
321
322
# Configure CORS for web applications
323
cors_rule = CorsRule(
324
allowed_origins="https://myapp.example.com,https://admin.example.com",
325
allowed_methods="GET,POST,PUT,DELETE",
326
max_age_in_seconds=3600,
327
exposed_headers="x-ms-request-id,x-ms-version",
328
allowed_headers="x-ms-client-request-id,x-custom-header"
329
)
330
331
# Apply service configuration
332
service_client.set_service_properties(
333
analytics_logging=logging_config,
334
hour_metrics=metrics_config,
335
minute_metrics=metrics_config,
336
cors=[cors_rule]
337
)
338
```
339
340
### Client-Side Encryption
341
342
Client-side encryption support for securing message content before transmission to Azure Storage.
343
344
```python { .api }
345
class CustomerProvidedEncryptionKey:
346
"""Customer-provided encryption key for client-side encryption."""
347
348
def __init__(
349
self,
350
key_value: Union[str, bytes],
351
key_hash: Optional[str] = None
352
):
353
"""
354
Create customer-provided encryption key.
355
356
Parameters:
357
- key_value: The encryption key value (base64 string or bytes)
358
- key_hash: SHA256 hash of the key (auto-calculated if not provided)
359
"""
360
361
key_value: str
362
key_hash: str
363
364
class EncryptionScope:
365
"""Encryption scope for server-side encryption."""
366
367
def __init__(self, encryption_scope: str):
368
"""
369
Create encryption scope.
370
371
Parameters:
372
- encryption_scope: Name of the encryption scope
373
"""
374
375
encryption_scope: str
376
```
377
378
### No-Encoding Policies
379
380
Policies for handling raw message content without encoding transformations.
381
382
```python { .api }
383
class NoEncodePolicy:
384
"""Policy that performs no encoding on message content."""
385
386
def encode(self, content: Any) -> Any:
387
"""
388
Return content without encoding.
389
390
Parameters:
391
- content: Message content to process
392
393
Returns:
394
Unmodified content
395
"""
396
397
class NoDecodePolicy:
398
"""Policy that performs no decoding on message content."""
399
400
def decode(self, content: Any, response) -> Any:
401
"""
402
Return content without decoding.
403
404
Parameters:
405
- content: Message content to process
406
- response: HTTP response object
407
408
Returns:
409
Unmodified content
410
"""
411
```
412
413
### Client-Side Encryption Usage
414
415
```python
416
from azure.storage.queue import QueueClient, CustomerProvidedEncryptionKey
417
import base64
418
import os
419
420
# Generate encryption key
421
key_bytes = os.urandom(32) # 256-bit key
422
key_base64 = base64.b64encode(key_bytes).decode('utf-8')
423
424
# Create encryption key object
425
encryption_key = CustomerProvidedEncryptionKey(key_value=key_base64)
426
427
# Create client with encryption
428
queue_client = QueueClient.from_connection_string(
429
conn_str,
430
"encrypted-queue",
431
customer_provided_encryption_key=encryption_key
432
)
433
434
# Send encrypted message
435
encrypted_message = queue_client.send_message("Sensitive data content")
436
print(f"Encrypted message sent: {encrypted_message.id}")
437
438
# Receive and decrypt message (requires same encryption key)
439
message = queue_client.receive_message()
440
if message:
441
print(f"Decrypted content: {message.content}")
442
queue_client.delete_message(message)
443
```
444
445
### No-Encoding Policy Usage
446
447
```python
448
from azure.storage.queue import QueueClient, NoEncodePolicy, NoDecodePolicy
449
import json
450
451
# Create client with no encoding policies for raw data handling
452
queue_client = QueueClient.from_connection_string(
453
conn_str,
454
"raw-data-queue",
455
message_encode_policy=NoEncodePolicy(),
456
message_decode_policy=NoDecodePolicy()
457
)
458
459
# Send raw JSON data
460
raw_data = {"timestamp": "2024-01-01T12:00:00Z", "value": 42}
461
json_string = json.dumps(raw_data)
462
queue_client.send_message(json_string)
463
464
# Receive raw data
465
message = queue_client.receive_message()
466
if message:
467
# message.content will be the raw JSON string
468
data = json.loads(message.content)
469
print(f"Received data: {data}")
470
queue_client.delete_message(message)
471
```
472
473
### Error Handling Patterns
474
475
```python
476
from azure.storage.queue import QueueClient, StorageErrorCode
477
from azure.core.exceptions import (
478
ResourceExistsError, ResourceNotFoundError,
479
ClientAuthenticationError, HttpResponseError
480
)
481
482
queue_client = QueueClient.from_connection_string(conn_str, "myqueue")
483
484
try:
485
queue_client.create_queue()
486
487
except ResourceExistsError as e:
488
if e.error_code == StorageErrorCode.QUEUE_ALREADY_EXISTS:
489
print("Queue already exists, continuing...")
490
else:
491
raise
492
493
try:
494
message = queue_client.receive_message()
495
if message:
496
# Process message
497
queue_client.delete_message(message)
498
499
except HttpResponseError as e:
500
if e.error_code == StorageErrorCode.MESSAGE_NOT_FOUND:
501
print("Message was already processed by another consumer")
502
elif e.error_code == StorageErrorCode.POP_RECEIPT_MISMATCH:
503
print("Message pop receipt is invalid, likely timed out")
504
else:
505
print(f"Unexpected error: {e.error_code}")
506
raise
507
508
try:
509
large_message = "x" * 100000 # > 64KB
510
queue_client.send_message(large_message)
511
512
except HttpResponseError as e:
513
if e.error_code == StorageErrorCode.REQUEST_BODY_TOO_LARGE:
514
print("Message too large, splitting into smaller parts...")
515
# Handle large message splitting logic
516
else:
517
raise
518
```
519
520
### Location Mode Usage
521
522
```python
523
from azure.storage.queue import QueueServiceClient, LocationMode
524
525
# Create client with secondary read access
526
service_client = QueueServiceClient.from_connection_string(
527
conn_str,
528
location_mode=LocationMode.SECONDARY # Read from secondary endpoint
529
)
530
531
# For read operations when primary is unavailable
532
try:
533
properties = service_client.get_service_properties()
534
except Exception:
535
# Fallback to secondary read
536
secondary_client = QueueServiceClient.from_connection_string(
537
conn_str,
538
location_mode=LocationMode.SECONDARY
539
)
540
properties = secondary_client.get_service_properties()
541
```
542
543
## Types
544
545
### Complete Type Definitions
546
547
```python { .api }
548
from typing import Optional, Dict, List, Any, Union, Callable
549
from datetime import datetime
550
from azure.core.paging import ItemPaged
551
552
# Message encoding function signatures
553
EncodeFunctionType = Callable[[Any], str]
554
DecodeFunctionType = Callable[[str, Any], Any]
555
556
# Configuration value constraints
557
MIN_RETENTION_DAYS = 1
558
MAX_RETENTION_DAYS = 365
559
MIN_CORS_MAX_AGE = 1
560
MAX_CORS_MAX_AGE = 2147483647
561
MIN_RETRY_BACKOFF = 0
562
MAX_RETRY_BACKOFF = 3600
563
MIN_RETRY_TOTAL = 0
564
MAX_RETRY_TOTAL = 10
565
566
# Analytics and metrics type unions
567
AnalyticsConfigType = Union[QueueAnalyticsLogging, Dict[str, Any], None]
568
MetricsConfigType = Union[Metrics, Dict[str, Any], None]
569
CorsConfigType = Union[List[CorsRule], List[Dict[str, Any]], None]
570
571
# Encryption types
572
EncryptionKeyType = Union[CustomerProvidedEncryptionKey, str, bytes, None]
573
EncryptionScopeType = Union[EncryptionScope, str, None]
574
575
# Encoding policy types
576
EncodePolicy = Union[TextBase64EncodePolicy, BinaryBase64EncodePolicy, NoEncodePolicy]
577
DecodePolicy = Union[TextBase64DecodePolicy, BinaryBase64DecodePolicy, NoDecodePolicy]
578
```