0
# OpenAI Integration
1

2
Automatic tracing for OpenAI SDK calls using a proxy wrapper. The `observeOpenAI` function wraps your OpenAI client to automatically capture all API calls with inputs, outputs, token usage, and timing information.
3

4
## Capabilities
5

6
### ObserveOpenAI Function
7

8
Wraps an OpenAI SDK instance with automatic Langfuse tracing.
9

10
```typescript { .api }
11
/**
12
 * Wraps an OpenAI SDK instance with automatic Langfuse tracing
13
 * @param sdk - The OpenAI SDK instance to wrap
14
 * @param langfuseConfig - Optional tracing configuration
15
 * @returns Wrapped SDK with tracing and lifecycle methods
16
 */
17
function observeOpenAI<SDKType extends object>(
18
  sdk: SDKType,
19
  langfuseConfig?: LangfuseConfig
20
): SDKType & LangfuseExtension;
21

22
interface LangfuseExtension {
23
  /**
24
   * Flushes all pending Langfuse events
25
   * @returns Promise that resolves when all events are sent
26
   */
27
  flushAsync(): Promise<void>;
28

29
  /**
30
   * Shuts down the Langfuse client
31
   * @returns Promise that resolves when shutdown is complete
32
   */
33
  shutdownAsync(): Promise<void>;
34
}
35
```
36

37
**Usage Example:**
38

39
```typescript
40
import OpenAI from 'openai';
41
import { observeOpenAI } from 'langfuse';
42

43
const client = new OpenAI({
44
  apiKey: process.env.OPENAI_API_KEY
45
});
46

47
// Wrap the client with tracing
48
const tracedClient = observeOpenAI(client, {
49
  traceName: 'openai-chat',
50
  userId: 'user-123',
51
  sessionId: 'session-456'
52
});
53

54
// Use normally - all calls are automatically traced
55
const response = await tracedClient.chat.completions.create({
56
  messages: [
57
    { role: 'system', content: 'You are a helpful assistant' },
58
    { role: 'user', content: 'Hello!' }
59
  ],
60
  model: 'gpt-3.5-turbo'
61
});
62

63
// Flush events to Langfuse
64
await tracedClient.flushAsync();
65
```
66

67
### Configuration Options
68

69
The `LangfuseConfig` type supports two modes: creating new traces or nesting under existing traces.
70

71
```typescript { .api }
72
type LangfuseConfig = (LangfuseNewTraceConfig | LangfuseWithParentConfig) & {
73
  /** Optional name for the generation */
74
  generationName?: string;
75
  /** Optional prompt client for linking */
76
  langfusePrompt?: LangfusePromptClient;
77
};
78

79
interface LangfuseNewTraceConfig {
80
  /** Custom trace ID */
81
  traceId?: string;
82
  /** Trace name */
83
  traceName?: string;
84
  /** Session ID */
85
  sessionId?: string;
86
  /** User ID */
87
  userId?: string;
88
  /** Release version */
89
  release?: string;
90
  /** Version identifier */
91
  version?: string;
92
  /** Custom metadata */
93
  metadata?: any;
94
  /** Tags for filtering */
95
  tags?: string[];
96
  /** Client initialization parameters */
97
  clientInitParams?: LangfuseInitParams;
98
}
99

100
interface LangfuseWithParentConfig {
101
  /** Parent trace, span, or generation to nest under */
102
  parent: LangfuseParent;
103
  /** Custom metadata */
104
  metadata?: any;
105
  /** Version identifier */
106
  version?: string;
107
  /** Prompt name (deprecated, use langfusePrompt) */
108
  promptName?: string;
109
  /** Prompt version (deprecated, use langfusePrompt) */
110
  promptVersion?: number;
111
}
112

113
type LangfuseParent =
114
  | LangfuseTraceClient
115
  | LangfuseSpanClient
116
  | LangfuseGenerationClient;
117

118
interface LangfuseInitParams {
119
  /** Langfuse public key */
120
  publicKey?: string;
121
  /** Langfuse secret key */
122
  secretKey?: string;
123
  /** Base URL for Langfuse API */
124
  baseUrl?: string;
125
  /** Additional Langfuse options */
126
  [key: string]: any;
127
}
128
```
129

130
## Usage Patterns
131

132
### Basic Tracing
133

134
Simple wrapper for automatic tracing of all OpenAI calls.
135

136
```typescript
137
import OpenAI from 'openai';
138
import { observeOpenAI } from 'langfuse';
139

140
const client = new OpenAI();
141
const tracedClient = observeOpenAI(client);
142

143
// All methods are automatically traced
144
const completion = await tracedClient.chat.completions.create({
145
  messages: [{ role: 'user', content: 'Hello!' }],
146
  model: 'gpt-3.5-turbo'
147
});
148

149
await tracedClient.flushAsync();
150
```
151

152
### Named Traces
153

154
Provide custom trace names for better organization.
155

156
```typescript
157
const tracedClient = observeOpenAI(client, {
158
  traceName: 'customer-support-chat',
159
  userId: 'user-123',
160
  sessionId: 'session-456',
161
  metadata: {
162
    environment: 'production',
163
    region: 'us-east-1'
164
  },
165
  tags: ['support', 'chat']
166
});
167

168
const response = await tracedClient.chat.completions.create({
169
  messages: [{ role: 'user', content: 'I need help' }],
170
  model: 'gpt-4'
171
});
172

173
await tracedClient.flushAsync();
174
```
175

176
### Custom Generation Names
177

178
Override the default generation name for specific calls.
179

180
```typescript
181
const tracedClient = observeOpenAI(client, {
182
  traceName: 'document-processing',
183
  generationName: 'document-summarization'
184
});
185

186
const summary = await tracedClient.chat.completions.create({
187
  messages: [
188
    { role: 'system', content: 'Summarize the following document' },
189
    { role: 'user', content: documentText }
190
  ],
191
  model: 'gpt-4'
192
});
193

194
await tracedClient.flushAsync();
195
```
196

197
### Nesting Under Existing Traces
198

199
Nest OpenAI calls under existing Langfuse traces for hierarchical tracing.
200

201
```typescript
202
import { Langfuse, observeOpenAI } from 'langfuse';
203

204
const langfuse = new Langfuse();
205
const client = new OpenAI();
206

207
// Create a parent trace
208
const trace = langfuse.trace({
209
  name: 'rag-pipeline',
210
  userId: 'user-123'
211
});
212

213
// Create a span for the retrieval step
214
const retrievalSpan = trace.span({
215
  name: 'document-retrieval'
216
});
217

218
// ... perform retrieval ...
219

220
retrievalSpan.end({
221
  output: { documents: retrievedDocs }
222
});
223

224
// Wrap OpenAI client to nest under the trace
225
const tracedClient = observeOpenAI(client, {
226
  parent: trace,
227
  generationName: 'answer-generation'
228
});
229

230
// OpenAI call will be nested under the trace
231
const response = await tracedClient.chat.completions.create({
232
  messages: [
233
    { role: 'system', content: 'Answer based on the context' },
234
    { role: 'user', content: query }
235
  ],
236
  model: 'gpt-4'
237
});
238

239
trace.update({
240
  output: { answer: response.choices[0].message.content }
241
});
242

243
await langfuse.flushAsync();
244
```
245

246
### Nesting Under Spans
247

248
Nest OpenAI calls under specific spans for detailed tracing.
249

250
```typescript
251
const trace = langfuse.trace({ name: 'multi-step-process' });
252

253
const step1Span = trace.span({ name: 'step-1' });
254

255
// Nest OpenAI call under this span
256
const tracedClient = observeOpenAI(client, {
257
  parent: step1Span,
258
  generationName: 'step-1-generation'
259
});
260

261
const result = await tracedClient.chat.completions.create({
262
  messages: [{ role: 'user', content: 'Step 1 prompt' }],
263
  model: 'gpt-3.5-turbo'
264
});
265

266
step1Span.end({
267
  output: { result: result.choices[0].message.content }
268
});
269

270
await langfuse.flushAsync();
271
```
272

273
### Linking Prompts
274

275
Link prompt templates to OpenAI generations for version tracking.
276

277
```typescript
278
import { Langfuse, observeOpenAI } from 'langfuse';
279

280
const langfuse = new Langfuse();
281
const client = new OpenAI();
282

283
// Fetch a prompt
284
const prompt = await langfuse.getPrompt('chat-template', undefined, {
285
  type: 'chat'
286
});
287

288
// Compile the prompt
289
const messages = prompt.compile(
290
  { topic: 'AI', tone: 'professional' },
291
  { history: [] }
292
);
293

294
// Wrap OpenAI with prompt linking
295
const tracedClient = observeOpenAI(client, {
296
  traceName: 'templated-chat',
297
  langfusePrompt: prompt
298
});
299

300
// The generation will be linked to the prompt version
301
const response = await tracedClient.chat.completions.create({
302
  messages: messages,
303
  model: 'gpt-4'
304
});
305

306
await tracedClient.flushAsync();
307
```
308

309
### Streaming Responses
310

311
Automatic tracing works with streaming responses.
312

313
```typescript
314
const tracedClient = observeOpenAI(client, {
315
  traceName: 'streaming-chat'
316
});
317

318
const stream = await tracedClient.chat.completions.create({
319
  messages: [{ role: 'user', content: 'Tell me a story' }],
320
  model: 'gpt-4',
321
  stream: true
322
});
323

324
let fullContent = '';
325

326
for await (const chunk of stream) {
327
  const content = chunk.choices[0]?.delta?.content || '';
328
  fullContent += content;
329
  process.stdout.write(content);
330
}
331

332
// Streaming responses are automatically captured
333
await tracedClient.flushAsync();
334
```
335

336
### Multiple OpenAI Calls
337

338
Each wrapped client can make multiple calls, all traced under the same configuration.
339

340
```typescript
341
const tracedClient = observeOpenAI(client, {
342
  traceName: 'multi-call-workflow',
343
  sessionId: 'session-789'
344
});
345

346
// First call
347
const classification = await tracedClient.chat.completions.create({
348
  messages: [
349
    { role: 'system', content: 'Classify the intent' },
350
    { role: 'user', content: userMessage }
351
  ],
352
  model: 'gpt-3.5-turbo'
353
});
354

355
// Second call (both under same trace)
356
const response = await tracedClient.chat.completions.create({
357
  messages: [
358
    { role: 'system', content: 'Generate response' },
359
    { role: 'user', content: userMessage }
360
  ],
361
  model: 'gpt-4'
362
});
363

364
await tracedClient.flushAsync();
365
```
366

367
### Embeddings and Other Methods
368

369
All OpenAI SDK methods are automatically traced, not just chat completions.
370

371
```typescript
372
const tracedClient = observeOpenAI(client, {
373
  traceName: 'embedding-pipeline'
374
});
375

376
// Embeddings are automatically traced
377
const embeddings = await tracedClient.embeddings.create({
378
  input: 'Text to embed',
379
  model: 'text-embedding-ada-002'
380
});
381

382
// Completions are traced
383
const completion = await tracedClient.completions.create({
384
  prompt: 'Once upon a time',
385
  model: 'gpt-3.5-turbo-instruct',
386
  max_tokens: 100
387
});
388

389
await tracedClient.flushAsync();
390
```
391

392
### Custom Client Initialization
393

394
Provide custom Langfuse client initialization parameters.
395

396
```typescript
397
const tracedClient = observeOpenAI(client, {
398
  traceName: 'custom-config',
399
  clientInitParams: {
400
    publicKey: 'custom-public-key',
401
    secretKey: 'custom-secret-key',
402
    baseUrl: 'https://custom-langfuse.com',
403
    flushAt: 1, // Flush after every event
404
    flushInterval: 5000
405
  }
406
});
407

408
const response = await tracedClient.chat.completions.create({
409
  messages: [{ role: 'user', content: 'Hello' }],
410
  model: 'gpt-3.5-turbo'
411
});
412

413
await tracedClient.flushAsync();
414
```
415

416
## Complete OpenAI Integration Example
417

418
```typescript
419
import OpenAI from 'openai';
420
import { Langfuse, observeOpenAI } from 'langfuse';
421

422
// Initialize clients
423
const langfuse = new Langfuse();
424
const openai = new OpenAI();
425

426
// Create a parent trace for the entire workflow
427
const trace = langfuse.trace({
428
  name: 'rag-qa-system',
429
  userId: 'user-456',
430
  sessionId: 'session-123',
431
  tags: ['production', 'qa']
432
});
433

434
// Step 1: Generate query embedding
435
const embeddingClient = observeOpenAI(openai, {
436
  parent: trace,
437
  generationName: 'query-embedding'
438
});
439

440
const queryEmbedding = await embeddingClient.embeddings.create({
441
  input: 'What is machine learning?',
442
  model: 'text-embedding-ada-002'
443
});
444

445
// Step 2: Retrieve documents (simulated)
446
const retrievalSpan = trace.span({
447
  name: 'document-retrieval',
448
  input: { query: 'What is machine learning?' }
449
});
450

451
const documents = await retrieveDocuments(queryEmbedding.data[0].embedding);
452

453
retrievalSpan.end({
454
  output: { documentCount: documents.length }
455
});
456

457
// Step 3: Generate answer with context
458
const prompt = await langfuse.getPrompt('qa-with-context', undefined, {
459
  type: 'chat'
460
});
461

462
const messages = prompt.compile(
463
  {
464
    context: documents.join('\n'),
465
    question: 'What is machine learning?'
466
  },
467
  { history: [] }
468
);
469

470
const answerClient = observeOpenAI(openai, {
471
  parent: trace,
472
  generationName: 'answer-generation',
473
  langfusePrompt: prompt
474
});
475

476
const answer = await answerClient.chat.completions.create({
477
  messages: messages,
478
  model: 'gpt-4',
479
  temperature: 0.7,
480
  max_tokens: 500
481
});
482

483
// Update trace with final output
484
trace.update({
485
  output: {
486
    answer: answer.choices[0].message.content,
487
    model: 'gpt-4',
488
    promptVersion: prompt.version
489
  }
490
});
491

492
// Add a quality score
493
trace.score({
494
  name: 'answer-quality',
495
  value: 0.95,
496
  comment: 'High quality answer with proper context'
497
});
498

499
// Flush all events
500
await langfuse.flushAsync();
501

502
// Get trace URL
503
console.log('View trace:', trace.getTraceUrl());
504
```
505

506
## Error Handling
507

508
The wrapper preserves errors from the OpenAI SDK while still capturing trace information.
509

510
```typescript
511
const tracedClient = observeOpenAI(client, {
512
  traceName: 'error-handling-example'
513
});
514

515
try {
516
  const response = await tracedClient.chat.completions.create({
517
    messages: [{ role: 'user', content: 'Hello' }],
518
    model: 'invalid-model' // This will cause an error
519
  });
520
} catch (error) {
521
  console.error('OpenAI error:', error);
522
  // Error is captured in the trace with ERROR level
523
} finally {
524
  await tracedClient.flushAsync();
525
}
526
```
527

528
## Performance Considerations
529

530
The `observeOpenAI` wrapper adds minimal overhead:
531

532
- **Proxy Pattern**: Uses JavaScript Proxy for transparent method interception
533
- **Async Tracing**: Events are queued and sent asynchronously without blocking OpenAI calls
534
- **Batching**: Multiple events are batched together for efficient network usage
535
- **Caching**: Prompt caching reduces API calls to Langfuse
536

537
**Best Practices:**
538

539
```typescript
540
// Good: Create one wrapper per workflow
541
const tracedClient = observeOpenAI(client, { traceName: 'workflow' });
542
await tracedClient.chat.completions.create(/* ... */);
543
await tracedClient.chat.completions.create(/* ... */);
544
await tracedClient.flushAsync();
545

546
// Avoid: Creating multiple wrappers for single calls
547
// This works but creates unnecessary overhead
548
const client1 = observeOpenAI(client, { traceName: 'call1' });
549
await client1.chat.completions.create(/* ... */);
550
await client1.flushAsync();
551

552
const client2 = observeOpenAI(client, { traceName: 'call2' });
553
await client2.chat.completions.create(/* ... */);
554
await client2.flushAsync();
555
```
556