0
# Configuration
1
2
Shared configuration options, utilities, and protocol objects used across all exporters. The configuration system provides centralized options management, connection string parsing, and common utilities for Azure Monitor integration.
3
4
## Capabilities
5
6
### Options Configuration
7
8
Central configuration class that manages all exporter options with validation and defaults.
9
10
```python { .api }
11
class Options(BaseObject):
12
"""
13
Configuration options for Azure exporters.
14
15
Provides centralized configuration with validation, defaults,
16
and Azure Monitor-specific option processing.
17
"""
18
19
def __init__(self, **options):
20
"""
21
Initialize configuration options.
22
23
Args:
24
**options: Configuration key-value pairs
25
26
Common Options:
27
connection_string (str): Azure Monitor connection string (recommended)
28
instrumentation_key (str): Instrumentation key (deprecated)
29
endpoint (str): Custom endpoint URL
30
export_interval (float): Export frequency in seconds (default: 15.0)
31
max_batch_size (int): Maximum batch size (default: 100)
32
enable_local_storage (bool): Enable local storage (default: True)
33
storage_path (str): Custom storage path
34
timeout (float): Network timeout in seconds (default: 10.0)
35
logging_sampling_rate (float): Log sampling rate (default: 1.0)
36
enable_standard_metrics (bool): Enable standard metrics (default: True)
37
credential: Azure credential for authentication
38
proxies (dict): Proxy configuration
39
"""
40
```
41
42
### Configuration Functions
43
44
Utility functions for processing and validating configuration options.
45
46
```python { .api }
47
def process_options(options):
48
"""
49
Process and validate configuration options.
50
51
Args:
52
options (dict): Raw configuration options
53
54
Returns:
55
dict: Processed and validated options
56
"""
57
58
def parse_connection_string(connection_string):
59
"""
60
Parse Azure Monitor connection string.
61
62
Args:
63
connection_string (str): Azure Monitor connection string
64
65
Returns:
66
dict: Parsed connection components including instrumentation_key
67
and ingestion_endpoint
68
69
Raises:
70
ValueError: If connection string format is invalid
71
"""
72
73
def validate_instrumentation_key(instrumentation_key):
74
"""
75
Validate instrumentation key format.
76
77
Args:
78
instrumentation_key (str): Instrumentation key to validate
79
80
Raises:
81
ValueError: If instrumentation key format is invalid
82
"""
83
```
84
85
### Configuration Constants
86
87
```python { .api }
88
INGESTION_ENDPOINT = "IngestionEndpoint"
89
INSTRUMENTATION_KEY = "InstrumentationKey"
90
TEMPDIR_PREFIX = "opencensus-python-"
91
```
92
93
## Usage Examples
94
95
### Basic Configuration
96
97
```python
98
from opencensus.ext.azure.log_exporter import AzureLogHandler
99
100
# Using connection string (recommended)
101
handler = AzureLogHandler(
102
connection_string="InstrumentationKey=your-instrumentation-key;IngestionEndpoint=https://your-region.in.applicationinsights.azure.com/"
103
)
104
105
# Using instrumentation key directly (deprecated)
106
handler = AzureLogHandler(
107
instrumentation_key="your-instrumentation-key"
108
)
109
```
110
111
### Advanced Configuration
112
113
```python
114
from opencensus.ext.azure.trace_exporter import AzureExporter
115
from opencensus.ext.azure.common.storage import LocalFileStorage
116
117
exporter = AzureExporter(
118
connection_string="InstrumentationKey=your-key-here",
119
export_interval=30.0, # Export every 30 seconds
120
max_batch_size=50, # Batch up to 50 items
121
timeout=15.0, # 15 second network timeout
122
enable_local_storage=True, # Enable persistence
123
storage_path="/tmp/azure_monitor", # Custom storage location
124
storage_max_size=50 * 1024 * 1024, # 50MB max storage
125
storage_retention_period=7 * 24 * 3600, # 7 days retention
126
proxies={ # Proxy configuration
127
'http': 'http://proxy.company.com:8080',
128
'https': 'https://proxy.company.com:8080'
129
}
130
)
131
```
132
133
### Authentication with Azure Identity
134
135
```python
136
from azure.identity import DefaultAzureCredential
137
from opencensus.ext.azure.metrics_exporter import new_metrics_exporter
138
139
# Use Azure Identity for authentication
140
credential = DefaultAzureCredential()
141
142
exporter = new_metrics_exporter(
143
connection_string="InstrumentationKey=your-key-here",
144
credential=credential
145
)
146
```
147
148
### Environment-Based Configuration
149
150
```python
151
import os
152
from opencensus.ext.azure.log_exporter import AzureLogHandler
153
154
# Configuration from environment variables
155
handler = AzureLogHandler(
156
connection_string=os.environ.get('AZURE_MONITOR_CONNECTION_STRING'),
157
export_interval=float(os.environ.get('EXPORT_INTERVAL', '15.0')),
158
max_batch_size=int(os.environ.get('MAX_BATCH_SIZE', '100')),
159
enable_local_storage=os.environ.get('ENABLE_STORAGE', 'true').lower() == 'true'
160
)
161
```
162
163
## Configuration Options Reference
164
165
### Connection Options
166
167
- **`connection_string`** (str): Complete Azure Monitor connection string containing instrumentation key and endpoint
168
- Format: `"InstrumentationKey=<key>;IngestionEndpoint=<endpoint>"`
169
- Recommended over separate `instrumentation_key` and `endpoint`
170
171
- **`instrumentation_key`** (str): Azure Application Insights instrumentation key
172
- **Deprecated**: Use `connection_string` instead
173
- Format: GUID string (e.g., "12345678-1234-1234-1234-123456789012")
174
175
- **`endpoint`** (str): Custom ingestion endpoint URL
176
- Default: Derived from connection string or standard Azure endpoint
177
- Format: Full URL (e.g., "https://your-region.in.applicationinsights.azure.com/")
178
179
- **`credential`**: Azure credential object for authentication
180
- Use with Azure Identity library for AAD authentication
181
- Alternative to instrumentation key for enhanced security
182
183
### Export Behavior Options
184
185
- **`export_interval`** (float): How often to export telemetry in seconds
186
- Default: 15.0
187
- Range: 1.0 to 3600.0 (1 second to 1 hour)
188
- Lower values provide more real-time data but increase overhead
189
190
- **`max_batch_size`** (int): Maximum number of telemetry items per batch
191
- Default: 100
192
- Range: 1 to 1000
193
- Higher values reduce HTTP overhead but increase memory usage
194
195
- **`timeout`** (float): Network request timeout in seconds
196
- Default: 10.0
197
- Range: 1.0 to 60.0
198
- Balance between responsiveness and reliability
199
200
### Storage Options
201
202
- **`enable_local_storage`** (bool): Enable local file persistence
203
- Default: True
204
- Provides reliability during network outages
205
- Telemetry is stored locally and retried when connectivity resumes
206
207
- **`storage_path`** (str): Directory for local storage files
208
- Default: System temp directory with prefix
209
- Must be writable by the application process
210
211
- **`storage_max_size`** (int): Maximum storage directory size in bytes
212
- Default: 50 MB (50 * 1024 * 1024)
213
- Oldest files are deleted when limit is exceeded
214
215
- **`storage_maintenance_period`** (int): Storage cleanup frequency in seconds
216
- Default: 86400 (24 hours)
217
- How often to clean up old storage files
218
219
- **`storage_retention_period`** (int): Storage file retention time in seconds
220
- Default: 604800 (7 days)
221
- Files older than this are automatically deleted
222
223
### Sampling and Filtering Options
224
225
- **`logging_sampling_rate`** (float): Sampling rate for log records
226
- Default: 1.0 (100% sampling)
227
- Range: 0.0 to 1.0
228
- Only applies to log exporters
229
230
- **`enable_standard_metrics`** (bool): Enable automatic standard metrics
231
- Default: True
232
- Only applies to metrics exporters
233
- Collects CPU, memory, and request metrics
234
235
### Network Options
236
237
- **`proxies`** (dict): HTTP proxy configuration
238
- Format: `{'http': 'http://proxy:port', 'https': 'https://proxy:port'}`
239
- Supports authentication: `'http://user:pass@proxy:port'`
240
241
### Advanced Options
242
243
- **`grace_period`** (float): Shutdown grace period in seconds
244
- Default: 5.0
245
- Time to wait for clean shutdown during process termination
246
247
- **`queue_capacity`** (int): Maximum queued telemetry items
248
- Default: 8192
249
- Prevents memory exhaustion under high load
250
251
- **`minimum_retry_interval`** (int): Minimum retry delay in seconds
252
- Default: 60
253
- Initial delay before retrying failed exports
254
255
## Connection String Format
256
257
Azure Monitor connection strings contain multiple components:
258
259
```
260
InstrumentationKey=12345678-1234-1234-1234-123456789012;IngestionEndpoint=https://your-region.in.applicationinsights.azure.com/;LiveEndpoint=https://your-region.livediagnostics.monitor.azure.com/
261
```
262
263
### Required Components
264
265
- **InstrumentationKey**: Your Application Insights instrumentation key
266
- **IngestionEndpoint**: Regional ingestion endpoint for telemetry
267
268
### Optional Components
269
270
- **LiveEndpoint**: Endpoint for live metrics and diagnostics
271
- **AADAudience**: Azure Active Directory audience for authentication
272
273
## Validation and Error Handling
274
275
The configuration system performs validation:
276
277
- **Instrumentation Key**: Must be valid GUID format
278
- **Connection String**: Must contain required components
279
- **Numeric Values**: Must be within valid ranges
280
- **File Paths**: Must be accessible for storage options
281
282
Common configuration errors:
283
284
```python
285
# Null or empty instrumentation key
286
ValueError: "Instrumentation key cannot be none or empty."
287
288
# Invalid instrumentation key format
289
ValueError: "Invalid instrumentation key."
290
291
# Invalid sampling rate
292
ValueError: "Sampling must be in the range: [0,1]"
293
294
# Invalid batch size
295
ValueError: "Max batch size must be at least 1."
296
```
297
298
### Protocol Classes
299
300
Azure Monitor protocol objects used for telemetry data structures.
301
302
```python { .api }
303
class BaseObject(dict):
304
"""
305
Base class for all Azure Monitor protocol objects.
306
307
Provides attribute-style access to dictionary data with defaults.
308
"""
309
310
def __init__(self, *args, **kwargs):
311
"""
312
Initialize the base object.
313
314
Args:
315
*args: Positional arguments passed to dict
316
**kwargs: Keyword arguments set as attributes
317
"""
318
319
class Envelope(BaseObject):
320
"""
321
Telemetry envelope containing metadata and data payload.
322
323
Standard wrapper for all telemetry items sent to Azure Monitor.
324
"""
325
326
def __init__(self, *args, **kwargs):
327
"""
328
Initialize telemetry envelope.
329
330
Args:
331
ver (int): Schema version (default: 1)
332
name (str): Telemetry item name
333
time (str): Timestamp in ISO format
334
iKey (str): Instrumentation key
335
tags (dict): Context tags
336
data (Data): Telemetry data payload
337
"""
338
339
class Data(BaseObject):
340
"""
341
Telemetry data container holding base data and type information.
342
"""
343
344
def __init__(self, *args, **kwargs):
345
"""
346
Initialize data container.
347
348
Args:
349
baseData: The actual telemetry data object
350
baseType (str): Type identifier for the base data
351
"""
352
353
class DataPoint(BaseObject):
354
"""
355
Metric data point containing value and metadata.
356
"""
357
358
def __init__(self, *args, **kwargs):
359
"""
360
Initialize metric data point.
361
362
Args:
363
ns (str): Namespace
364
name (str): Metric name
365
value (float): Metric value
366
kind (str, optional): Data point kind
367
count (int, optional): Sample count
368
min (float, optional): Minimum value
369
max (float, optional): Maximum value
370
stdDev (float, optional): Standard deviation
371
"""
372
373
class Event(BaseObject):
374
"""
375
Custom event telemetry data.
376
"""
377
378
def __init__(self, *args, **kwargs):
379
"""
380
Initialize event data.
381
382
Args:
383
name (str): Event name
384
properties (dict): Custom properties
385
measurements (dict): Custom measurements
386
"""
387
388
class ExceptionData(BaseObject):
389
"""
390
Exception telemetry data with stack trace information.
391
"""
392
393
def __init__(self, *args, **kwargs):
394
"""
395
Initialize exception data.
396
397
Args:
398
exceptions (list): List of exception details
399
severityLevel (int): Exception severity level
400
properties (dict): Custom properties
401
"""
402
403
class Message(BaseObject):
404
"""
405
Log message telemetry data.
406
"""
407
408
def __init__(self, *args, **kwargs):
409
"""
410
Initialize message data.
411
412
Args:
413
message (str): Log message text
414
severityLevel (int): Message severity level
415
properties (dict): Custom properties
416
"""
417
418
class MetricData(BaseObject):
419
"""
420
Metric telemetry data container.
421
"""
422
423
def __init__(self, *args, **kwargs):
424
"""
425
Initialize metric data.
426
427
Args:
428
metrics (list): List of DataPoint objects
429
properties (dict): Custom properties
430
"""
431
432
class Request(BaseObject):
433
"""
434
HTTP request telemetry data.
435
"""
436
437
def __init__(self, *args, **kwargs):
438
"""
439
Initialize request data.
440
441
Args:
442
id (str): Request identifier
443
name (str): Request name
444
url (str): Request URL
445
duration (str): Request duration
446
responseCode (str): HTTP response code
447
success (bool): Request success status
448
properties (dict): Custom properties
449
"""
450
451
class RemoteDependency(BaseObject):
452
"""
453
Remote dependency call telemetry data.
454
"""
455
456
def __init__(self, *args, **kwargs):
457
"""
458
Initialize dependency data.
459
460
Args:
461
id (str): Dependency call identifier
462
name (str): Dependency name
463
type (str): Dependency type (HTTP, SQL, etc.)
464
target (str): Dependency target
465
data (str): Dependency data (URL, command, etc.)
466
duration (str): Call duration
467
resultCode (str): Result code
468
success (bool): Call success status
469
properties (dict): Custom properties
470
"""
471
```
472
473
### Storage Classes
474
475
Local file storage system for reliable telemetry persistence.
476
477
```python { .api }
478
class LocalFileStorage:
479
"""
480
Local file storage for telemetry persistence during network issues.
481
482
Provides reliable storage with automatic maintenance, retention,
483
and retry capabilities for Azure Monitor telemetry.
484
"""
485
486
def __init__(self, path, max_size=50*1024*1024, maintenance_period=60,
487
retention_period=7*24*60*60, write_timeout=60, source=None):
488
"""
489
Initialize local file storage.
490
491
Args:
492
path (str): Storage directory path
493
max_size (int): Maximum storage size in bytes (default: 50MB)
494
maintenance_period (int): Cleanup interval in seconds (default: 60)
495
retention_period (int): File retention time in seconds (default: 7 days)
496
write_timeout (int): Write operation timeout in seconds (default: 60)
497
source (str, optional): Source identifier for maintenance task naming
498
"""
499
500
def put(self, data, lease_period=0):
501
"""
502
Store telemetry data to local file.
503
504
Args:
505
data (list): List of telemetry items to store
506
lease_period (int): Lease time in seconds (default: 0)
507
508
Returns:
509
LocalFileBlob: Blob object for stored data, or None if storage full
510
"""
511
512
def get(self):
513
"""
514
Retrieve next available telemetry blob.
515
516
Returns:
517
LocalFileBlob: Next available blob, or None if none available
518
"""
519
520
def gets(self):
521
"""
522
Generator for all available telemetry blobs.
523
524
Yields:
525
LocalFileBlob: Available blob objects in chronological order
526
"""
527
528
def close(self):
529
"""
530
Close storage and stop maintenance task.
531
"""
532
533
class LocalFileBlob:
534
"""
535
Individual telemetry data file with lease capabilities.
536
537
Represents a single stored telemetry batch with operations
538
for reading, leasing, and cleanup.
539
"""
540
541
def __init__(self, fullpath):
542
"""
543
Initialize file blob.
544
545
Args:
546
fullpath (str): Full path to the blob file
547
"""
548
549
def get(self):
550
"""
551
Read telemetry data from blob file.
552
553
Returns:
554
tuple: Tuple of JSON-decoded telemetry items, or None on error
555
"""
556
557
def put(self, data, lease_period=0):
558
"""
559
Write telemetry data to blob file.
560
561
Args:
562
data (list): List of telemetry items to write
563
lease_period (int): Initial lease time in seconds
564
565
Returns:
566
LocalFileBlob: Self reference, or None on error
567
"""
568
569
def lease(self, period):
570
"""
571
Acquire or extend lease on blob file.
572
573
Args:
574
period (int): Lease duration in seconds
575
576
Returns:
577
LocalFileBlob: Self reference if successful, None if failed
578
"""
579
580
def delete(self):
581
"""
582
Delete the blob file.
583
"""
584
```
585
586
### Utility Functions
587
588
Common utilities for Azure Monitor integration.
589
590
```python { .api }
591
def validate_instrumentation_key(instrumentation_key):
592
"""
593
Validate instrumentation key format.
594
595
Ensures the instrumentation key is not null/empty and matches
596
the required UUID format for Azure Monitor.
597
598
Args:
599
instrumentation_key (str): Instrumentation key to validate
600
601
Raises:
602
ValueError: If key is null, empty, or invalid format
603
"""
604
605
def parse_connection_string(connection_string):
606
"""
607
Parse Azure Monitor connection string into components.
608
609
Args:
610
connection_string (str): Connection string to parse
611
612
Returns:
613
dict: Parsed connection components with lowercase keys
614
615
Raises:
616
ValueError: If connection string format is invalid
617
"""
618
619
def timestamp_to_duration(start_time, end_time):
620
"""
621
Convert start and end timestamps to Azure Monitor duration format.
622
623
Args:
624
start_time: Start timestamp
625
end_time: End timestamp
626
627
Returns:
628
str: Duration in format 'd.hh:mm:ss.fff'
629
"""
630
631
def timestamp_to_iso_str(timestamp):
632
"""
633
Convert timestamp to ISO format string.
634
635
Args:
636
timestamp (float): Unix timestamp
637
638
Returns:
639
str: ISO formatted timestamp string
640
"""
641
642
def microseconds_to_duration(microseconds):
643
"""
644
Convert microseconds to Azure Monitor duration format.
645
646
Args:
647
microseconds (int): Duration in microseconds
648
649
Returns:
650
str: Duration in format 'd.hh:mm:ss.fff'
651
"""
652
653
# Azure Monitor context information
654
azure_monitor_context = {
655
'ai.cloud.role': 'Application role name',
656
'ai.cloud.roleInstance': 'Role instance identifier',
657
'ai.device.id': 'Device identifier',
658
'ai.device.locale': 'Device locale',
659
'ai.device.osVersion': 'Operating system version',
660
'ai.device.type': 'Device type',
661
'ai.internal.sdkVersion': 'SDK version information'
662
}
663
```
664
665
### Transport Layer
666
667
HTTP transport functionality with retry logic and status handling.
668
669
```python { .api }
670
class TransportStatusCode:
671
"""
672
Status codes for telemetry transmission results.
673
"""
674
SUCCESS = 0 # All telemetry successfully transmitted
675
RETRY = 1 # Transmission failed, should retry later
676
DROP = 2 # Transmission failed, should not retry
677
STATSBEAT_SHUTDOWN = 3 # Statsbeat shutdown signal
678
679
class TransportMixin:
680
"""
681
Mixin providing HTTP transport functionality for exporters.
682
683
Handles telemetry transmission to Azure Monitor with retry logic,
684
local storage integration, and error handling.
685
"""
686
687
def _transmit(self, envelopes):
688
"""
689
Transmit telemetry envelopes to Azure Monitor.
690
691
Args:
692
envelopes (list): List of telemetry envelopes to transmit
693
694
Returns:
695
int: TransportStatusCode indicating transmission result
696
"""
697
698
def _transmit_from_storage(self):
699
"""
700
Transmit telemetry from local storage.
701
702
Processes stored telemetry files and attempts retransmission,
703
cleaning up successfully sent items and re-leasing failed items.
704
"""
705
706
def _check_stats_collection(self):
707
"""
708
Check if statsbeat collection should be enabled.
709
710
Returns:
711
bool: True if statsbeat collection should be enabled
712
"""
713
714
# HTTP status code constants
715
RETRYABLE_STATUS_CODES = (401, 403, 408, 429, 500, 502, 503, 504)
716
THROTTLE_STATUS_CODES = (402, 439)
717
REDIRECT_STATUS_CODES = (307, 308)
718
```
719
720
### Processor Interface
721
722
Telemetry processing pipeline for filtering and enrichment.
723
724
```python { .api }
725
class ProcessorMixin:
726
"""
727
Mixin providing telemetry processor functionality.
728
729
Enables registration and application of custom telemetry processors
730
for filtering, enrichment, and modification of telemetry data.
731
"""
732
733
def add_telemetry_processor(self, processor):
734
"""
735
Add a telemetry processor to the processing pipeline.
736
737
Args:
738
processor (callable): Function that takes an envelope and returns
739
modified envelope or None to drop
740
"""
741
742
def apply_telemetry_processors(self, envelopes):
743
"""
744
Apply all registered processors to telemetry envelopes.
745
746
Args:
747
envelopes (list): List of telemetry envelopes to process
748
749
Returns:
750
list: Processed envelopes (may be filtered/modified)
751
"""
752
```
753
754
## Best Practices
755
756
1. **Use Connection Strings**: Prefer connection strings over separate key/endpoint
757
2. **Environment Variables**: Store sensitive configuration in environment variables
758
3. **Enable Local Storage**: Provides reliability during network issues
759
4. **Adjust Batch Size**: Optimize for your telemetry volume and latency requirements
760
5. **Monitor Storage Usage**: Ensure adequate disk space for local storage
761
6. **Use Azure Identity**: Leverage AAD authentication when possible
762
7. **Configure Proxies**: Set up proxy configuration for corporate environments