0
# OpenTelemetry Integration
1
2
OpenTelemetry API compatibility layer enables seamless interoperability between ddtrace and OpenTelemetry instrumentation while maintaining Datadog-specific features and optimizations. This allows applications to use OpenTelemetry APIs while benefiting from Datadog's advanced APM capabilities.
3
4
## Capabilities
5
6
### TracerProvider
7
8
The main entry point for OpenTelemetry integration, providing OpenTelemetry-compatible tracer instances that use ddtrace as the backend implementation.
9
10
```python { .api }
11
class TracerProvider:
12
def get_tracer(
13
self,
14
instrumenting_module_name: str,
15
instrumenting_library_version: str = None,
16
schema_url: str = None
17
) -> Tracer:
18
"""
19
Get an OpenTelemetry-compatible tracer instance.
20
21
Parameters:
22
- instrumenting_module_name: Name of the instrumenting module
23
- instrumenting_library_version: Version of the instrumenting library
24
- schema_url: Schema URL for telemetry data
25
26
Returns:
27
OpenTelemetry Tracer instance backed by ddtrace
28
"""
29
30
def add_span_processor(self, span_processor) -> None:
31
"""
32
Add a span processor to the tracer provider.
33
34
Parameters:
35
- span_processor: OpenTelemetry span processor
36
"""
37
38
def get_active_span_processor(self):
39
"""
40
Get the active span processor.
41
42
Returns:
43
Active span processor instance
44
"""
45
46
def shutdown(self) -> bool:
47
"""
48
Shutdown the tracer provider and flush pending spans.
49
50
Returns:
51
True if shutdown was successful
52
"""
53
```
54
55
### Configuration and Setup
56
57
Enable OpenTelemetry support and configure the TracerProvider for use with ddtrace backend.
58
59
```python
60
import os
61
62
# Enable OpenTelemetry support (must be set before importing ddtrace)
63
os.environ["DD_TRACE_OTEL_ENABLED"] = "true"
64
65
# Configure the OpenTelemetry TracerProvider
66
from opentelemetry.trace import set_tracer_provider
67
from ddtrace.opentelemetry import TracerProvider
68
69
# Set ddtrace as the OpenTelemetry backend
70
set_tracer_provider(TracerProvider())
71
72
# Now OpenTelemetry APIs will use ddtrace backend
73
from opentelemetry import trace
74
75
tracer = trace.get_tracer(__name__)
76
```
77
78
### Mixed API Usage
79
80
Combine OpenTelemetry and ddtrace APIs seamlessly within the same application.
81
82
```python
83
import os
84
os.environ["DD_TRACE_OTEL_ENABLED"] = "true"
85
86
from opentelemetry.trace import set_tracer_provider
87
from ddtrace.opentelemetry import TracerProvider
88
from opentelemetry import trace
89
import ddtrace
90
91
# Setup
92
set_tracer_provider(TracerProvider())
93
otel_tracer = trace.get_tracer(__name__)
94
95
# Mixed usage example
96
with otel_tracer.start_as_current_span("otel-parent") as parent_span:
97
# Set OpenTelemetry attributes
98
parent_span.set_attribute("service.name", "mixed-service")
99
parent_span.set_attribute("operation.type", "data-processing")
100
101
# Create ddtrace child span within OpenTelemetry parent
102
with ddtrace.tracer.trace("ddtrace-child") as child_span:
103
child_span.set_tag("component", "business-logic")
104
child_span.set_tag("user.id", "12345")
105
106
# Both spans are part of the same trace
107
process_business_logic()
108
109
# Back to OpenTelemetry span
110
parent_span.set_attribute("result.status", "success")
111
```
112
113
### OpenTelemetry Span Operations
114
115
Use standard OpenTelemetry span APIs with ddtrace backend providing the implementation.
116
117
```python
118
from opentelemetry import trace
119
120
tracer = trace.get_tracer(__name__)
121
122
# Context manager usage
123
with tracer.start_as_current_span("http-request") as span:
124
# Set standard OpenTelemetry attributes
125
span.set_attribute("http.method", "GET")
126
span.set_attribute("http.url", "https://api.example.com/users")
127
span.set_attribute("http.status_code", 200)
128
129
# Add events
130
span.add_event("request.started")
131
132
response = make_http_request()
133
134
span.add_event("request.completed", {
135
"response.size": len(response.content)
136
})
137
138
# Manual span management
139
span = tracer.start_span("database-operation")
140
try:
141
span.set_attribute("db.system", "postgresql")
142
span.set_attribute("db.statement", "SELECT * FROM users")
143
144
result = execute_database_query()
145
146
span.set_attribute("db.rows_affected", len(result))
147
span.set_status(trace.Status(trace.StatusCode.OK))
148
149
except Exception as e:
150
span.set_status(trace.Status(trace.StatusCode.ERROR, str(e)))
151
span.record_exception(e)
152
raise
153
finally:
154
span.end()
155
```
156
157
### Span Decorator Usage
158
159
Use OpenTelemetry decorators for automatic span creation.
160
161
```python
162
from opentelemetry import trace
163
164
tracer = trace.get_tracer(__name__)
165
166
@tracer.start_as_current_span("user-authentication")
167
def authenticate_user(username, password):
168
# Function automatically wrapped in a span
169
span = trace.get_current_span()
170
span.set_attribute("user.name", username)
171
span.set_attribute("auth.method", "password")
172
173
try:
174
user = validate_credentials(username, password)
175
span.set_attribute("auth.success", True)
176
span.set_attribute("user.id", user.id)
177
return user
178
except AuthenticationError as e:
179
span.set_attribute("auth.success", False)
180
span.set_attribute("error.type", "authentication_failed")
181
span.record_exception(e)
182
raise
183
184
# Usage
185
user = authenticate_user("john_doe", "password123")
186
```
187
188
### Attribute and Tag Mapping
189
190
Understanding how OpenTelemetry attributes map to Datadog tags enables effective use of both APIs.
191
192
```python
193
with tracer.start_as_current_span("mapped-operation") as span:
194
# OpenTelemetry attributes become Datadog tags
195
span.set_attribute("service.name", "payment-service") # -> service.name tag
196
span.set_attribute("http.method", "POST") # -> http.method tag
197
span.set_attribute("user.id", "12345") # -> user.id tag
198
span.set_attribute("custom.business_metric", "high_value") # -> custom.business_metric tag
199
200
# OpenTelemetry span kind maps to Datadog span.kind
201
span.set_attribute("span.kind", "client") # -> span.kind tag
202
203
# OpenTelemetry status maps to Datadog error state
204
span.set_status(trace.Status(trace.StatusCode.ERROR, "Payment failed"))
205
```
206
207
### Resource and Service Configuration
208
209
Configure service information through OpenTelemetry Resource API or ddtrace configuration.
210
211
```python
212
from opentelemetry.sdk.resources import Resource
213
from opentelemetry.semconv.resource import ResourceAttributes
214
215
# OpenTelemetry resource configuration
216
resource = Resource.create({
217
ResourceAttributes.SERVICE_NAME: "payment-processor",
218
ResourceAttributes.SERVICE_VERSION: "2.1.0",
219
ResourceAttributes.DEPLOYMENT_ENVIRONMENT: "production",
220
"custom.team": "payments",
221
"custom.region": "us-east-1"
222
})
223
224
# TracerProvider with resource configuration
225
provider = TracerProvider(resource=resource)
226
set_tracer_provider(provider)
227
228
# Alternatively, use ddtrace configuration
229
import ddtrace
230
231
ddtrace.config.service = "payment-processor"
232
ddtrace.config.version = "2.1.0"
233
ddtrace.config.env = "production"
234
ddtrace.config.tags = {
235
"team": "payments",
236
"region": "us-east-1"
237
}
238
```
239
240
### Instrumentation Integration
241
242
OpenTelemetry automatic instrumentation works seamlessly with ddtrace backend.
243
244
```python
245
import os
246
os.environ["DD_TRACE_OTEL_ENABLED"] = "true"
247
248
# OpenTelemetry auto-instrumentation
249
from opentelemetry.instrumentation.requests import RequestsInstrumentor
250
from opentelemetry.instrumentation.psycopg2 import Psycopg2Instrumentor
251
252
# Auto-instrument libraries with OpenTelemetry
253
RequestsInstrumentor().instrument()
254
Psycopg2Instrumentor().instrument()
255
256
# Set ddtrace as backend
257
from opentelemetry.trace import set_tracer_provider
258
from ddtrace.opentelemetry import TracerProvider
259
260
set_tracer_provider(TracerProvider())
261
262
# Now requests and psycopg2 calls are automatically traced
263
import requests
264
import psycopg2
265
266
# HTTP request traced via OpenTelemetry instrumentation -> ddtrace backend
267
response = requests.get("https://api.example.com/data")
268
269
# Database query traced via OpenTelemetry instrumentation -> ddtrace backend
270
conn = psycopg2.connect("dbname=mydb")
271
cursor = conn.cursor()
272
cursor.execute("SELECT * FROM users")
273
```
274
275
### Context Propagation
276
277
OpenTelemetry context propagation works with ddtrace trace context.
278
279
```python
280
from opentelemetry import trace, propagate
281
from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator
282
283
tracer = trace.get_tracer(__name__)
284
285
# Inject trace context into HTTP headers
286
with tracer.start_as_current_span("parent-service") as span:
287
headers = {}
288
289
# Extract current context and inject into headers
290
propagate.inject(headers)
291
292
# Headers now contain trace context for downstream services
293
response = requests.get(
294
"https://downstream-service.com/api",
295
headers=headers
296
)
297
298
# Extract trace context from incoming request
299
def handle_request(request_headers):
300
# Extract context from incoming headers
301
ctx = propagate.extract(request_headers)
302
303
# Start span with extracted context as parent
304
with tracer.start_as_current_span("downstream-operation", context=ctx) as span:
305
span.set_attribute("service.role", "downstream")
306
return process_request()
307
```
308
309
### Advanced OpenTelemetry Features
310
311
#### Span Links
312
313
```python
314
from opentelemetry import trace
315
from opentelemetry.trace import Link
316
317
tracer = trace.get_tracer(__name__)
318
319
# Create span with links to related spans
320
with tracer.start_as_current_span("batch-processor") as batch_span:
321
batch_span.set_attribute("batch.size", 100)
322
323
item_spans = []
324
for i in range(5): # Process sample items
325
with tracer.start_as_current_span(f"process-item-{i}") as item_span:
326
item_span.set_attribute("item.index", i)
327
item_spans.append(item_span.get_span_context())
328
329
# Create summary span linked to all item spans
330
links = [Link(span_context) for span_context in item_spans]
331
with tracer.start_as_current_span("batch-summary", links=links) as summary_span:
332
summary_span.set_attribute("summary.type", "batch_completion")
333
```
334
335
#### Span Events
336
337
```python
338
with tracer.start_as_current_span("file-processing") as span:
339
span.add_event("file.opened", {
340
"file.name": "data.csv",
341
"file.size": 1024000
342
})
343
344
data = read_file("data.csv")
345
346
span.add_event("file.read", {
347
"rows.count": len(data),
348
"processing.duration": 0.5
349
})
350
351
results = process_data(data)
352
353
span.add_event("processing.completed", {
354
"results.count": len(results),
355
"success.rate": 0.95
356
})
357
```
358
359
### Performance Considerations
360
361
OpenTelemetry integration with ddtrace maintains high performance while providing API compatibility.
362
363
```python
364
# Efficient span creation
365
with tracer.start_as_current_span("high-frequency-operation") as span:
366
# Minimize attribute operations in hot paths
367
span.set_attribute("operation.id", operation_id)
368
369
# Batch attribute setting when possible
370
span.set_attributes({
371
"service.component": "data-processor",
372
"operation.type": "transform",
373
"batch.size": batch_size
374
})
375
376
result = perform_operation()
377
378
# Set result attributes
379
span.set_attribute("result.status", "success")
380
```
381
382
### Migration from Pure OpenTelemetry
383
384
Migrate existing OpenTelemetry applications to use ddtrace backend with minimal code changes.
385
386
```python
387
# Before: Pure OpenTelemetry setup
388
# from opentelemetry.sdk.trace import TracerProvider
389
# from opentelemetry.sdk.trace.export import BatchSpanProcessor
390
# from opentelemetry.exporter.jaeger.thrift import JaegerExporter
391
392
# After: OpenTelemetry with ddtrace backend
393
import os
394
os.environ["DD_TRACE_OTEL_ENABLED"] = "true"
395
396
from opentelemetry.trace import set_tracer_provider
397
from ddtrace.opentelemetry import TracerProvider
398
399
# Replace OpenTelemetry SDK with ddtrace backend
400
set_tracer_provider(TracerProvider())
401
402
# Rest of the application code remains unchanged
403
from opentelemetry import trace
404
405
tracer = trace.get_tracer(__name__)
406
407
with tracer.start_as_current_span("migrated-operation") as span:
408
span.set_attribute("migration.status", "completed")
409
# Existing OpenTelemetry code works without changes
410
```
411
412
## Data Mapping
413
414
OpenTelemetry concepts map to Datadog concepts as follows:
415
416
- **trace_id** → **traceID**
417
- **span_id** → **spanID**
418
- **parent_span_id** → **parentID**
419
- **name** → **resource**
420
- **kind** → **meta["span.kind"]**
421
- **start_time_unix_nano** → **start**
422
- **end_time_unix_nano** → **duration** (derived)
423
- **attributes[<key>]** → **meta[<key>]**
424
- **links[]** → **meta["_dd.span_links"]**
425
- **status** → **error** (derived)
426
- **events[]** → **meta["events"]**
427
428
This mapping ensures that OpenTelemetry instrumentation produces Datadog-compatible traces with all semantic information preserved.