0
# AWS Service Integration
1

2
Specialized integration for AWS services through boto3/botocore patching. Automatically traces AWS API calls with service-specific metadata, error handling, and comprehensive request/response information for effective monitoring of AWS service interactions.
3

4
## Capabilities
5

6
### Automatic AWS Service Tracing
7

8
Enable automatic tracing of all AWS service calls through boto3/botocore patching.
9

10
```python { .api }
11
# Import and patch botocore for AWS service tracing
12
from aws_xray_sdk.core import patch
13
patch(['botocore'])
14

15
# For async AWS calls
16
patch(['aiobotocore'])
17
```
18

19
### Service Call Metadata
20

21
AWS service calls automatically include comprehensive metadata:
22

23
- **Service Information**: AWS service name, operation, region
24
- **Request Details**: API operation, parameters (sanitized)
25
- **Response Metadata**: Request ID, status, response size
26
- **Error Information**: AWS error codes, error messages, retry information
27
- **Timing Data**: Request/response timing, connection time
28

29
## Supported AWS Services
30

31
All AWS services supported by boto3/botocore are automatically traced, including:
32

33
### Compute Services
34
- **EC2**: Instance management, security groups, networking
35
- **Lambda**: Function invocation, layer management
36
- **ECS**: Container management, task execution
37
- **Batch**: Job management, compute environments
38

39
### Storage Services
40
- **S3**: Object operations, bucket management, multipart uploads
41
- **EBS**: Volume management, snapshot operations
42
- **EFS**: File system operations
43
- **Glacier**: Archive operations, vault management
44

45
### Database Services
46
- **DynamoDB**: Table operations, item management, queries, scans
47
- **RDS**: Database management, backup operations
48
- **ElastiCache**: Cache cluster management
49
- **DocumentDB**: Document database operations
50

51
### Messaging and Queuing
52
- **SQS**: Queue operations, message handling
53
- **SNS**: Topic management, message publishing
54
- **EventBridge**: Event rule management, event publishing
55
- **Kinesis**: Stream operations, data processing
56

57
### Security and Identity
58
- **IAM**: User and role management, policy operations
59
- **Cognito**: User pool management, identity operations
60
- **Secrets Manager**: Secret management and retrieval
61
- **Systems Manager**: Parameter store, patch management
62

63
### Monitoring and Logging
64
- **CloudWatch**: Metrics, alarms, log group management
65
- **X-Ray**: Trace management (when not using SDK)
66
- **CloudTrail**: Audit trail management
67

68
## Usage Patterns
69

70
### Basic AWS Service Tracing
71

72
```python
73
from aws_xray_sdk.core import patch, xray_recorder
74
import boto3
75

76
# Patch botocore for automatic AWS tracing
77
patch(['botocore'])
78

79
# Configure X-Ray
80
xray_recorder.configure(
81
    service='AWS Integration Demo',
82
    daemon_address='127.0.0.1:2000'
83
)
84

85
with xray_recorder.in_segment('aws-operations') as segment:
86
    # All AWS calls are automatically traced
87
    
88
    # S3 operations
89
    s3 = boto3.client('s3')
90
    buckets = s3.list_buckets()  # Automatically traced as subsegment
91
    
92
    # DynamoDB operations
93
    dynamodb = boto3.resource('dynamodb')
94
    table = dynamodb.Table('users')
95
    response = table.get_item(Key={'id': '123'})  # Automatically traced
96
    
97
    # Lambda invocation
98
    lambda_client = boto3.client('lambda')
99
    response = lambda_client.invoke(
100
        FunctionName='my-function',
101
        Payload=json.dumps({'key': 'value'})
102
    )  # Automatically traced
103
```
104

105
### Service-Specific Operations
106

107
#### S3 Operations
108

109
```python
110
from aws_xray_sdk.core import patch, xray_recorder
111
import boto3
112

113
patch(['botocore'])
114

115
with xray_recorder.in_segment('s3-operations') as segment:
116
    s3 = boto3.client('s3')
117
    
118
    # Each operation creates a subsegment with detailed metadata
119
    
120
    # Upload file - traces upload progress for large files
121
    s3.upload_file('local-file.txt', 'my-bucket', 'remote-file.txt')
122
    
123
    # Download file - traces download progress
124
    s3.download_file('my-bucket', 'remote-file.txt', 'downloaded-file.txt')
125
    
126
    # List objects - includes pagination tracing
127
    response = s3.list_objects_v2(Bucket='my-bucket', Prefix='data/')
128
    
129
    # Multipart upload - traces each part
130
    response = s3.create_multipart_upload(Bucket='my-bucket', Key='large-file.dat')
131
    upload_id = response['UploadId']
132
    
133
    # Each part upload is traced separately
134
    part1 = s3.upload_part(
135
        Bucket='my-bucket',
136
        Key='large-file.dat',
137
        PartNumber=1,
138
        UploadId=upload_id,
139
        Body=b'part1 data'
140
    )
141
```
142

143
#### DynamoDB Operations
144

145
```python
146
from aws_xray_sdk.core import patch, xray_recorder
147
import boto3
148

149
patch(['botocore'])
150

151
with xray_recorder.in_segment('dynamodb-operations') as segment:
152
    # Resource interface
153
    dynamodb = boto3.resource('dynamodb')
154
    table = dynamodb.Table('users')
155
    
156
    # Item operations - automatically traced with table metadata
157
    response = table.put_item(
158
        Item={'id': '123', 'name': 'John Doe', 'email': 'john@example.com'}
159
    )
160
    
161
    user = table.get_item(Key={'id': '123'})
162
    
163
    # Query operations - includes index and pagination info
164
    response = table.query(
165
        IndexName='email-index',
166
        KeyConditionExpression=Key('email').eq('john@example.com')
167
    )
168
    
169
    # Scan operations - includes parallel scan metadata
170
    response = table.scan(
171
        FilterExpression=Attr('active').eq(True),
172
        Limit=100
173
    )
174
    
175
    # Batch operations - traces batch efficiency
176
    with table.batch_writer() as batch:
177
        for i in range(100):
178
            batch.put_item(Item={'id': str(i), 'data': f'item{i}'})
179
```
180

181
#### Lambda Operations
182

183
```python
184
from aws_xray_sdk.core import patch, xray_recorder
185
import boto3
186
import json
187

188
patch(['botocore'])
189

190
with xray_recorder.in_segment('lambda-operations') as segment:
191
    lambda_client = boto3.client('lambda')
192
    
193
    # Synchronous invocation - includes execution time and response
194
    response = lambda_client.invoke(
195
        FunctionName='data-processor',
196
        InvocationType='RequestResponse',
197
        Payload=json.dumps({'data': 'process this'})
198
    )
199
    result = json.loads(response['Payload'].read())
200
    
201
    # Asynchronous invocation - traces invocation only
202
    lambda_client.invoke(
203
        FunctionName='async-processor',
204
        InvocationType='Event',
205
        Payload=json.dumps({'data': 'process async'})
206
    )
207
    
208
    # List functions - includes pagination
209
    functions = lambda_client.list_functions(MaxItems=50)
210
```
211

212
### Error Handling and Retries
213

214
```python
215
from aws_xray_sdk.core import patch, xray_recorder
216
import boto3
217
from botocore.exceptions import ClientError
218

219
patch(['botocore'])
220

221
with xray_recorder.in_segment('aws-error-handling') as segment:
222
    s3 = boto3.client('s3')
223
    
224
    try:
225
        # Operation that might fail
226
        response = s3.get_object(Bucket='non-existent-bucket', Key='file.txt')
227
        
228
    except ClientError as e:
229
        # AWS errors are automatically captured with:
230
        # - Error code (NoSuchBucket, AccessDenied, etc.)
231
        # - Error message 
232
        # - Request ID for AWS support
233
        # - HTTP status code
234
        # - Retry information
235
        
236
        error_code = e.response['Error']['Code']
237
        xray_recorder.put_annotation('aws_error_code', error_code)
238
        
239
        if error_code == 'NoSuchBucket':
240
            # Handle specific error
241
            segment.add_error_flag()  # Mark as client error
242
        else:
243
            # Handle other errors
244
            segment.add_fault_flag()  # Mark as service fault
245
        
246
        raise
247
```
248

249
### Batch Operations and Pagination
250

251
```python
252
from aws_xray_sdk.core import patch, xray_recorder
253
import boto3
254

255
patch(['botocore'])
256

257
with xray_recorder.in_segment('batch-operations') as segment:
258
    # DynamoDB batch operations
259
    dynamodb = boto3.resource('dynamodb')
260
    
261
    with xray_recorder.in_subsegment('batch-write') as subsegment:
262
        # Batch write automatically handles retries and traces each batch
263
        with dynamodb.batch_writer() as batch:
264
            for i in range(1000):  # Automatically batched into 25-item chunks
265
                batch.put_item(Item={'id': str(i), 'data': f'item{i}'})
266
        
267
        subsegment.put_annotation('items_written', '1000')
268
    
269
    # S3 paginated operations
270
    s3 = boto3.client('s3')
271
    
272
    with xray_recorder.in_subsegment('list-all-objects') as subsegment:
273
        paginator = s3.get_paginator('list_objects_v2')
274
        
275
        object_count = 0
276
        for page in paginator.paginate(Bucket='my-large-bucket'):
277
            # Each page request is traced as a separate subsegment
278
            object_count += len(page.get('Contents', []))
279
        
280
        subsegment.put_annotation('total_objects', str(object_count))
281
```
282

283
### Async AWS Operations
284

285
```python
286
from aws_xray_sdk.core import patch, xray_recorder
287
from aws_xray_sdk.core.async_context import AsyncContext
288
import aiobotocore.session
289
import asyncio
290

291
# Patch aiobotocore for async AWS tracing
292
patch(['aiobotocore'])
293

294
# Configure recorder with async context
295
xray_recorder.configure(
296
    service='Async AWS Demo',
297
    context=AsyncContext()
298
)
299

300
async def async_aws_operations():
301
    async with xray_recorder.in_segment_async('async-aws-ops') as segment:
302
        session = aiobotocore.session.get_session()
303
        
304
        # Async S3 operations
305
        async with session.create_client('s3') as s3:
306
            # Each async operation is traced
307
            buckets = await s3.list_buckets()
308
            
309
            # Concurrent operations are traced individually
310
            tasks = [
311
                s3.head_object(Bucket='my-bucket', Key=f'file{i}.txt')
312
                for i in range(10)
313
            ]
314
            results = await asyncio.gather(*tasks, return_exceptions=True)
315
        
316
        # Async DynamoDB operations
317
        async with session.create_client('dynamodb') as dynamodb:
318
            response = await dynamodb.describe_table(TableName='users')
319
            
320
            # Batch async operations
321
            put_requests = [
322
                dynamodb.put_item(
323
                    TableName='users',
324
                    Item={'id': {'S': str(i)}, 'name': {'S': f'User{i}'}}
325
                )
326
                for i in range(50)
327
            ]
328
            await asyncio.gather(*put_requests)
329

330
# Run async operations
331
asyncio.run(async_aws_operations())
332
```
333

334
### Custom Metadata and Annotations
335

336
```python
337
from aws_xray_sdk.core import patch, xray_recorder
338
import boto3
339

340
patch(['botocore'])
341

342
with xray_recorder.in_segment('custom-aws-metadata') as segment:
343
    # Add custom annotations for filtering
344
    xray_recorder.put_annotation('environment', 'production')
345
    xray_recorder.put_annotation('service_version', '1.2.3')
346
    
347
    dynamodb = boto3.resource('dynamodb')
348
    table = dynamodb.Table('user_sessions')
349
    
350
    with xray_recorder.in_subsegment('user-session-cleanup') as subsegment:
351
        # Add business context
352
        subsegment.put_annotation('cleanup_type', 'expired_sessions')
353
        subsegment.put_annotation('table_name', 'user_sessions')
354
        
355
        # Scan for expired sessions
356
        response = table.scan(
357
            FilterExpression=Attr('expires_at').lt(int(time.time())),
358
            ProjectionExpression='session_id'
359
        )
360
        
361
        expired_sessions = response['Items']
362
        subsegment.put_annotation('expired_count', str(len(expired_sessions)))
363
        
364
        # Add detailed metadata
365
        subsegment.put_metadata('cleanup_info', {
366
            'total_scanned': response['ScannedCount'],
367
            'expired_found': len(expired_sessions),
368
            'consumed_capacity': response.get('ConsumedCapacity', {}),
369
            'scan_time': time.time()
370
        }, namespace='cleanup')
371
        
372
        # Batch delete expired sessions
373
        if expired_sessions:
374
            with table.batch_writer() as batch:
375
                for session in expired_sessions:
376
                    batch.delete_item(Key={'session_id': session['session_id']})
377
```
378

379
## Advanced Features
380

381
### Custom AWS Metadata Processing
382

383
```python
384
from aws_xray_sdk.core import patch, xray_recorder
385

386
# Custom metadata processor for AWS calls
387
def aws_metadata_processor(wrapped, instance, args, kwargs, name, namespace, meta_processor):
388
    # Add custom logic before/after AWS calls
389
    start_time = time.time()
390
    
391
    try:
392
        result = wrapped(*args, **kwargs)
393
        
394
        # Add custom success metadata
395
        xray_recorder.put_metadata('aws_call_timing', {
396
            'duration_ms': (time.time() - start_time) * 1000,
397
            'success': True
398
        }, namespace='custom')
399
        
400
        return result
401
        
402
    except Exception as e:
403
        # Add custom error metadata
404
        xray_recorder.put_metadata('aws_call_error', {
405
            'duration_ms': (time.time() - start_time) * 1000,
406
            'error_type': type(e).__name__,
407
            'error_message': str(e)
408
        }, namespace='custom')
409
        raise
410

411
patch(['botocore'])
412
```
413

414
### Service-Specific Configuration
415

416
```python
417
from aws_xray_sdk.core import patch, xray_recorder
418

419
# Configure different services differently
420
xray_recorder.configure(
421
    service='Multi-Service App',
422
    sampling=True,
423
    sampling_rules='sampling-rules.json'
424
)
425

426
# Custom sampling rules can target specific AWS services
427
# Example sampling-rules.json:
428
# {
429
#   "rules": [
430
#     {
431
#       "description": "High-volume DynamoDB operations",
432
#       "service_name": "*",
433
#       "http_method": "*",
434
#       "url_path": "*",
435
#       "fixed_target": 1,
436
#       "rate": 0.1,
437
#       "service_type": "AWS::DynamoDB::Table"
438
#     },
439
#     {
440
#       "description": "S3 operations",
441
#       "service_name": "*", 
442
#       "http_method": "*",
443
#       "url_path": "*",
444
#       "fixed_target": 2,
445
#       "rate": 0.5,
446
#       "service_type": "AWS::S3::Bucket"
447
#     }
448
#   ]
449
# }
450

451
patch(['botocore'])
452
```
453

454
## Best Practices
455

456
### Performance Optimization
457

458
```python
459
# Use sampling for high-volume AWS operations
460
xray_recorder.configure(sampling=True)
461

462
# Conditional expensive metadata
463
with xray_recorder.in_segment('aws-operations') as segment:
464
    if xray_recorder.is_sampled():
465
        # Only add expensive metadata for sampled traces
466
        segment.put_metadata('detailed_aws_config', get_detailed_aws_config())
467
    
468
    # Lightweight annotations for all traces
469
    xray_recorder.put_annotation('aws_region', 'us-east-1')
470
```
471

472
### Security Considerations
473

474
```python
475
# AWS SDK automatically sanitizes sensitive data
476
# But you can add additional sanitization
477

478
def sanitize_aws_parameters(params):
479
    """Remove sensitive information from AWS parameters."""
480
    sensitive_keys = ['password', 'secret', 'token', 'key']
481
    
482
    if isinstance(params, dict):
483
        return {
484
            k: '***REDACTED***' if any(sensitive in k.lower() for sensitive in sensitive_keys) 
485
            else sanitize_aws_parameters(v)
486
            for k, v in params.items()
487
        }
488
    elif isinstance(params, list):
489
        return [sanitize_aws_parameters(item) for item in params]
490
    else:
491
        return params
492

493
# Use in custom metadata
494
xray_recorder.put_metadata('sanitized_params', sanitize_aws_parameters(aws_params))
495
```
496

497
### Error Handling Patterns
498

499
```python
500
from botocore.exceptions import ClientError, NoCredentialsError
501

502
with xray_recorder.in_segment('robust-aws-operations') as segment:
503
    try:
504
        # AWS operations
505
        result = perform_aws_operations()
506
        
507
    except NoCredentialsError:
508
        # Credential issues
509
        segment.add_fault_flag()
510
        xray_recorder.put_annotation('error_category', 'credentials')
511
        raise
512
        
513
    except ClientError as e:
514
        error_code = e.response['Error']['Code']
515
        
516
        if error_code in ['ThrottlingException', 'ProvisionedThroughputExceededException']:
517
            # Throttling - may want to retry
518
            segment.add_throttle_flag()
519
            xray_recorder.put_annotation('error_category', 'throttling')
520
        elif error_code.startswith('4'):
521
            # Client errors
522
            segment.add_error_flag()
523
            xray_recorder.put_annotation('error_category', 'client_error')
524
        else:
525
            # Service errors
526
            segment.add_fault_flag()
527
            xray_recorder.put_annotation('error_category', 'service_error')
528
        
529
        raise
530
```