0
# Modes and Configuration
1
2
The instructor package provides a comprehensive mode system for configuring how structured extraction works across different LLM providers. Each mode optimizes for specific provider capabilities and use cases.
3
4
## Mode Enumeration
5
6
The Mode enum defines extraction strategies for different providers and scenarios.
7
8
```python { .api }
9
from instructor import Mode
10
11
class Mode(Enum):
12
"""
13
LLM interaction modes for structured extraction.
14
15
Defines how the instructor client should interact with different
16
LLM providers to achieve structured output extraction.
17
"""
18
19
# OpenAI Modes
20
TOOLS = "tools"
21
TOOLS_STRICT = "tools_strict"
22
JSON = "json"
23
JSON_O1 = "json_o1"
24
JSON_SCHEMA = "json_schema"
25
MD_JSON = "markdown_json"
26
PARALLEL_TOOLS = "parallel_tools"
27
FUNCTIONS = "functions" # Deprecated
28
29
# Response API Modes
30
RESPONSES_TOOLS = "responses_tools"
31
RESPONSES_TOOLS_WITH_INBUILT_TOOLS = "responses_tools_with_inbuilt_tools"
32
33
# XAI Modes
34
XAI_JSON = "xai_json"
35
XAI_TOOLS = "xai_tools"
36
37
# Anthropic Modes
38
ANTHROPIC_TOOLS = "anthropic_tools"
39
ANTHROPIC_JSON = "anthropic_json"
40
ANTHROPIC_REASONING_TOOLS = "anthropic_reasoning_tools"
41
ANTHROPIC_PARALLEL_TOOLS = "anthropic_parallel_tools"
42
43
# Provider-Specific Modes
44
MISTRAL_TOOLS = "mistral_tools"
45
VERTEXAI_TOOLS = "vertexai_tools"
46
GEMINI_TOOLS = "gemini_tools"
47
COHERE_TOOLS = "cohere_tools"
48
GROQ_TOOLS = "groq_tools"
49
FIREWORKS_TOOLS = "fireworks_tools"
50
CEREBRAS_TOOLS = "cerebras_tools"
51
WRITER_TOOLS = "writer_tools"
52
BEDROCK_TOOLS = "bedrock_tools"
53
54
@classmethod
55
def tool_modes(cls) -> List['Mode']:
56
"""
57
Get all tool-based modes.
58
59
Returns:
60
List of modes that use function/tool calling
61
"""
62
63
@classmethod
64
def json_modes(cls) -> List['Mode']:
65
"""
66
Get all JSON-based modes.
67
68
Returns:
69
List of modes that use JSON schema extraction
70
"""
71
72
@classmethod
73
def warn_mode_functions_deprecation(cls) -> None:
74
"""Warn about deprecated FUNCTIONS mode."""
75
```
76
77
## OpenAI Modes
78
79
### TOOLS Mode (Recommended)
80
81
Default and recommended mode for OpenAI models with function calling support.
82
83
```python { .api }
84
import instructor
85
from openai import OpenAI
86
87
# Standard tools mode
88
client = instructor.from_openai(
89
OpenAI(),
90
mode=instructor.Mode.TOOLS
91
)
92
93
# Usage
94
result = client.create(
95
model="gpt-4",
96
messages=[{"role": "user", "content": "Extract user: John, 25, engineer"}],
97
response_model=UserProfile
98
)
99
```
100
101
#### Characteristics:
102
- Uses OpenAI's function calling API
103
- Supports streaming and partial results
104
- Best reliability and type safety
105
- Compatible with GPT-4, GPT-3.5-turbo, and newer models
106
107
### TOOLS_STRICT Mode
108
109
Strict mode with enhanced schema validation and enforcement.
110
111
```python { .api }
112
# Strict tools mode for maximum reliability
113
client = instructor.from_openai(
114
OpenAI(),
115
mode=instructor.Mode.TOOLS_STRICT
116
)
117
118
# Enhanced validation and schema enforcement
119
result = client.create(
120
model="gpt-4",
121
messages=[{"role": "user", "content": "Extract data..."}],
122
response_model=StrictModel,
123
strict=True # Enforces strict schema adherence
124
)
125
```
126
127
#### Characteristics:
128
- Strict schema validation
129
- Enhanced error handling
130
- Reduced hallucination in function calls
131
- Optimal for production environments
132
133
### JSON Mode
134
135
Direct JSON schema extraction without function calling.
136
137
```python { .api }
138
# JSON mode for models without function calling
139
client = instructor.from_openai(
140
OpenAI(),
141
mode=instructor.Mode.JSON
142
)
143
144
# Forces JSON response format
145
result = client.create(
146
model="gpt-3.5-turbo-1106",
147
messages=[{"role": "user", "content": "Return JSON: user data..."}],
148
response_model=UserProfile,
149
response_format={"type": "json_object"}
150
)
151
```
152
153
#### Characteristics:
154
- Works with models lacking function calling
155
- Relies on JSON schema in prompt
156
- Less reliable than tools mode
157
- Useful for cost optimization
158
159
### JSON_O1 Mode
160
161
Specialized JSON mode optimized for OpenAI O1 models.
162
163
```python { .api }
164
# Optimized for O1 models
165
client = instructor.from_openai(
166
OpenAI(),
167
mode=instructor.Mode.JSON_O1
168
)
169
170
result = client.create(
171
model="o1-preview",
172
messages=[{"role": "user", "content": "Analyze and extract..."}],
173
response_model=AnalysisResult
174
)
175
```
176
177
#### Characteristics:
178
- Optimized for O1 model capabilities
179
- Enhanced reasoning integration
180
- Specialized prompt formatting
181
- Better performance on complex extractions
182
183
### PARALLEL_TOOLS Mode
184
185
Parallel function calling for multiple simultaneous extractions.
186
187
```python { .api }
188
# Parallel extraction mode
189
client = instructor.from_openai(
190
OpenAI(),
191
mode=instructor.Mode.PARALLEL_TOOLS
192
)
193
194
# Can extract multiple entities in one call
195
results = client.create(
196
model="gpt-4",
197
messages=[{"role": "user", "content": "Extract all users and products..."}],
198
response_model=List[Union[UserProfile, ProductInfo]]
199
)
200
```
201
202
#### Characteristics:
203
- Extracts multiple entities simultaneously
204
- Improved efficiency for batch operations
205
- Requires compatible models
206
- Complex result handling
207
208
## Anthropic Modes
209
210
### ANTHROPIC_TOOLS Mode
211
212
Primary mode for Anthropic models using their tool calling system.
213
214
```python { .api }
215
import instructor
216
from anthropic import Anthropic
217
218
# Standard Anthropic tools
219
client = instructor.from_anthropic(
220
Anthropic(),
221
mode=instructor.Mode.ANTHROPIC_TOOLS
222
)
223
224
result = client.create(
225
model="claude-3-sonnet-20240229",
226
messages=[{"role": "user", "content": "Extract information..."}],
227
response_model=DataModel
228
)
229
```
230
231
#### Characteristics:
232
- Native Anthropic tool integration
233
- High reliability and accuracy
234
- Supports complex nested structures
235
- Optimized for Claude 3+ models
236
237
### ANTHROPIC_JSON Mode
238
239
JSON-based extraction for Anthropic models.
240
241
```python { .api }
242
# JSON mode for Anthropic
243
client = instructor.from_anthropic(
244
Anthropic(),
245
mode=instructor.Mode.ANTHROPIC_JSON
246
)
247
248
result = client.create(
249
model="claude-3-haiku-20240307",
250
messages=[{"role": "user", "content": "Return structured data..."}],
251
response_model=SimpleModel
252
)
253
```
254
255
#### Characteristics:
256
- JSON schema-based extraction
257
- Works with all Claude models
258
- Cost-effective option
259
- Slightly less reliable than tools mode
260
261
### ANTHROPIC_REASONING_TOOLS Mode
262
263
Enhanced mode with reasoning capabilities (beta feature).
264
265
```python { .api }
266
# Reasoning tools mode (beta)
267
client = instructor.from_anthropic(
268
Anthropic(),
269
mode=instructor.Mode.ANTHROPIC_REASONING_TOOLS,
270
beta=True
271
)
272
273
result = client.create(
274
model="claude-3-opus-20240229",
275
messages=[{"role": "user", "content": "Analyze and extract insights..."}],
276
response_model=ReasoningResult,
277
reasoning=True
278
)
279
```
280
281
#### Characteristics:
282
- Includes reasoning traces
283
- Enhanced analytical capabilities
284
- Beta feature requiring opt-in
285
- Higher token usage
286
287
### ANTHROPIC_PARALLEL_TOOLS Mode
288
289
Parallel tool calling for Anthropic models.
290
291
```python { .api }
292
# Parallel tools for Anthropic
293
client = instructor.from_anthropic(
294
Anthropic(),
295
mode=instructor.Mode.ANTHROPIC_PARALLEL_TOOLS
296
)
297
298
results = client.create(
299
model="claude-3-sonnet-20240229",
300
messages=[{"role": "user", "content": "Extract multiple entities..."}],
301
response_model=List[EntityType]
302
)
303
```
304
305
## Provider-Specific Modes
306
307
### MISTRAL_TOOLS Mode
308
309
Optimized for Mistral AI models with tool calling.
310
311
```python { .api }
312
import instructor
313
from mistralai.client import MistralClient
314
315
client = instructor.from_mistral(
316
MistralClient(),
317
mode=instructor.Mode.MISTRAL_TOOLS
318
)
319
320
result = client.create(
321
model="mistral-large-latest",
322
messages=[{"role": "user", "content": "Extract data..."}],
323
response_model=DataModel
324
)
325
```
326
327
### VERTEXAI_TOOLS Mode
328
329
Google Vertex AI integration with function calling.
330
331
```python { .api }
332
import instructor
333
from vertexai.preview.generative_models import GenerativeModel
334
335
client = instructor.from_vertexai(
336
GenerativeModel("gemini-pro"),
337
mode=instructor.Mode.VERTEXAI_TOOLS
338
)
339
```
340
341
### GEMINI_TOOLS Mode
342
343
Google Gemini models with function calling support.
344
345
```python { .api }
346
import instructor
347
import google.generativeai as genai
348
349
client = instructor.from_genai(
350
genai.GenerativeModel("gemini-pro"),
351
mode=instructor.Mode.GEMINI_TOOLS
352
)
353
```
354
355
## Mode Selection Guidelines
356
357
### By Provider and Model
358
359
```python { .api }
360
def select_optimal_mode(provider: str, model: str) -> instructor.Mode:
361
"""
362
Select optimal mode based on provider and model capabilities.
363
364
Args:
365
provider: LLM provider name
366
model: Specific model identifier
367
368
Returns:
369
Recommended mode for the provider/model combination
370
"""
371
372
if provider == "openai":
373
if model.startswith("gpt-4"):
374
return instructor.Mode.TOOLS_STRICT
375
elif model.startswith("gpt-3.5-turbo"):
376
return instructor.Mode.TOOLS
377
elif model.startswith("o1"):
378
return instructor.Mode.JSON_O1
379
else:
380
return instructor.Mode.JSON
381
382
elif provider == "anthropic":
383
if "opus" in model or "sonnet" in model:
384
return instructor.Mode.ANTHROPIC_TOOLS
385
else:
386
return instructor.Mode.ANTHROPIC_JSON
387
388
elif provider == "mistral":
389
return instructor.Mode.MISTRAL_TOOLS
390
391
elif provider == "google":
392
if "gemini" in model:
393
return instructor.Mode.GEMINI_TOOLS
394
else:
395
return instructor.Mode.VERTEXAI_TOOLS
396
397
# Default fallback
398
return instructor.Mode.JSON
399
400
# Usage
401
optimal_mode = select_optimal_mode("openai", "gpt-4-turbo")
402
client = instructor.from_openai(OpenAI(), mode=optimal_mode)
403
```
404
405
### By Use Case
406
407
```python { .api }
408
# Production reliability - use strict modes
409
production_client = instructor.from_openai(
410
OpenAI(),
411
mode=instructor.Mode.TOOLS_STRICT
412
)
413
414
# Cost optimization - use JSON modes
415
cost_optimized_client = instructor.from_openai(
416
OpenAI(),
417
mode=instructor.Mode.JSON
418
)
419
420
# High throughput - use parallel modes
421
parallel_client = instructor.from_openai(
422
OpenAI(),
423
mode=instructor.Mode.PARALLEL_TOOLS
424
)
425
426
# Complex reasoning - use reasoning modes
427
reasoning_client = instructor.from_anthropic(
428
Anthropic(),
429
mode=instructor.Mode.ANTHROPIC_REASONING_TOOLS,
430
beta=True
431
)
432
```
433
434
## Configuration Options
435
436
### Client Configuration
437
438
```python { .api }
439
# Comprehensive client configuration
440
client = instructor.from_openai(
441
OpenAI(
442
api_key="your-key",
443
base_url="https://custom-endpoint.com",
444
timeout=30.0,
445
max_retries=3
446
),
447
mode=instructor.Mode.TOOLS_STRICT,
448
449
# Instructor-specific configuration
450
hooks=my_hooks,
451
max_validation_retries=2,
452
strict_mode=True,
453
enable_caching=True
454
)
455
```
456
457
### Mode-Specific Configuration
458
459
```python { .api }
460
# Tools mode with custom settings
461
tools_config = {
462
"strict": True,
463
"parallel_tool_calls": False,
464
"tool_choice": "auto"
465
}
466
467
client = instructor.from_openai(
468
OpenAI(),
469
mode=instructor.Mode.TOOLS_STRICT,
470
**tools_config
471
)
472
473
# JSON mode with custom settings
474
json_config = {
475
"response_format": {"type": "json_object"},
476
"temperature": 0.1,
477
"max_tokens": 1000
478
}
479
480
client = instructor.from_openai(
481
OpenAI(),
482
mode=instructor.Mode.JSON,
483
**json_config
484
)
485
```
486
487
### Dynamic Mode Switching
488
489
```python { .api }
490
class AdaptiveClient:
491
"""Client that adapts mode based on context."""
492
493
def __init__(self, base_client):
494
self.base_client = base_client
495
self.openai_client = instructor.from_openai(base_client)
496
497
def create_adaptive(
498
self,
499
model: str,
500
messages: List[Dict],
501
response_model: Type[BaseModel],
502
prefer_reliability: bool = True,
503
**kwargs
504
):
505
"""Create with adaptive mode selection."""
506
507
# Select mode based on context
508
if prefer_reliability:
509
if "gpt-4" in model:
510
mode = instructor.Mode.TOOLS_STRICT
511
else:
512
mode = instructor.Mode.TOOLS
513
else:
514
mode = instructor.Mode.JSON
515
516
# Recreate client with selected mode
517
adaptive_client = instructor.from_openai(
518
self.base_client,
519
mode=mode
520
)
521
522
return adaptive_client.create(
523
model=model,
524
messages=messages,
525
response_model=response_model,
526
**kwargs
527
)
528
529
# Usage
530
adaptive = AdaptiveClient(OpenAI())
531
532
# High reliability extraction
533
result = adaptive.create_adaptive(
534
model="gpt-4",
535
messages=[{"role": "user", "content": "Critical data..."}],
536
response_model=CriticalModel,
537
prefer_reliability=True
538
)
539
540
# Cost-optimized extraction
541
result = adaptive.create_adaptive(
542
model="gpt-3.5-turbo",
543
messages=[{"role": "user", "content": "Simple data..."}],
544
response_model=SimpleModel,
545
prefer_reliability=False
546
)
547
```
548
549
## Performance and Reliability
550
551
### Mode Comparison
552
553
| Mode | Reliability | Speed | Cost | Streaming | Parallel |
554
|------|------------|-------|------|-----------|----------|
555
| TOOLS | High | Fast | Medium | Yes | No |
556
| TOOLS_STRICT | Highest | Fast | Medium | Yes | No |
557
| JSON | Medium | Fastest | Low | Limited | No |
558
| JSON_O1 | High | Medium | High | No | No |
559
| PARALLEL_TOOLS | High | Medium | Medium | Yes | Yes |
560
| ANTHROPIC_TOOLS | High | Fast | Medium | Yes | No |
561
| ANTHROPIC_REASONING_TOOLS | Highest | Slow | High | Limited | No |
562
563
### Error Handling by Mode
564
565
```python { .api }
566
def robust_extraction(
567
client,
568
model: str,
569
messages: List[Dict],
570
response_model: Type[BaseModel],
571
fallback_modes: List[instructor.Mode] = None
572
):
573
"""Extraction with mode fallback on errors."""
574
575
if fallback_modes is None:
576
fallback_modes = [
577
instructor.Mode.TOOLS_STRICT,
578
instructor.Mode.TOOLS,
579
instructor.Mode.JSON
580
]
581
582
last_error = None
583
584
for mode in fallback_modes:
585
try:
586
# Recreate client with current mode
587
mode_client = instructor.from_openai(
588
client.client, # Access underlying OpenAI client
589
mode=mode
590
)
591
592
result = mode_client.create(
593
model=model,
594
messages=messages,
595
response_model=response_model
596
)
597
598
return result, mode
599
600
except Exception as e:
601
last_error = e
602
print(f"Mode {mode} failed: {e}")
603
continue
604
605
raise RuntimeError(f"All modes failed. Last error: {last_error}")
606
607
# Usage with fallback
608
try:
609
result, used_mode = robust_extraction(
610
client,
611
model="gpt-4",
612
messages=[{"role": "user", "content": "Extract..."}],
613
response_model=MyModel
614
)
615
print(f"Success with mode: {used_mode}")
616
except RuntimeError as e:
617
print(f"All extraction attempts failed: {e}")
618
```