0
# Chat Models
1

2
Advanced conversational AI capabilities using AWS Bedrock's Converse API with comprehensive support for streaming, function calling, multimodal input, structured output, and advanced features like reasoning content and prompt caching.
3

4
## Capabilities
5

6
### ChatBedrockConverse Class
7

8
Primary chat model class for AWS Bedrock Converse API integration, extending LangChain's BaseChatModel with AWS-specific capabilities.
9

10
```typescript { .api }
11
/**
12
 * AWS Bedrock Converse chat model integration with streaming and tool support
13
 */
14
class ChatBedrockConverse extends BaseChatModel<ChatBedrockConverseCallOptions, AIMessageChunk> {
15
  constructor(fields?: ChatBedrockConverseInput);
16
  
17
  /** Generate a single response from the model */
18
  invoke(messages: BaseMessage[], options?: ChatBedrockConverseCallOptions): Promise<AIMessage>;
19
  
20
  /** Stream response chunks for real-time output */
21
  stream(messages: BaseMessage[], options?: ChatBedrockConverseCallOptions): AsyncGenerator<AIMessageChunk>;
22
  
23
  /** Bind tools for function calling capabilities */
24
  bindTools(tools: ChatBedrockConverseToolType[], kwargs?: Partial<ChatBedrockConverseCallOptions>): Runnable<BaseLanguageModelInput, AIMessageChunk, ChatBedrockConverseCallOptions>;
25
  
26
  /** Force structured JSON output using schema validation */
27
  withStructuredOutput<T>(outputSchema: InteropZodType<T> | Record<string, any>, config?: StructuredOutputMethodOptions<boolean>): Runnable<BaseLanguageModelInput, T> | Runnable<BaseLanguageModelInput, { raw: BaseMessage; parsed: T }>;
28
  
29
  /** Get LangSmith parameters for tracing */
30
  getLsParams(options: ChatBedrockConverseCallOptions): LangSmithParams;
31
  
32
  /** Get invocation parameters for Bedrock API */
33
  invocationParams(options?: ChatBedrockConverseCallOptions): Partial<ConverseCommandParams>;
34
}
35
```
36

37
**Usage Examples:**
38

39
```typescript
40
import { ChatBedrockConverse } from "@langchain/aws";
41
import { HumanMessage } from "@langchain/core/messages";
42

43
// Basic initialization
44
const model = new ChatBedrockConverse({
45
  region: "us-east-1",
46
  model: "anthropic.claude-3-5-sonnet-20240620-v1:0",
47
  temperature: 0.7,
48
  maxTokens: 1000,
49
  credentials: {
50
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
51
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
52
  }
53
});
54

55
// Simple conversation
56
const response = await model.invoke([
57
  new HumanMessage("Explain quantum computing in simple terms")
58
]);
59

60
// Streaming conversation
61
const stream = await model.stream([
62
  new HumanMessage("Write a short story about a robot")
63
]);
64

65
for await (const chunk of stream) {
66
  process.stdout.write(chunk.content);
67
}
68
```
69

70
### Constructor Configuration
71

72
Comprehensive configuration options for initializing ChatBedrockConverse instances.
73

74
```typescript { .api }
75
interface ChatBedrockConverseInput extends BaseChatModelParams, Partial<DefaultProviderInit> {
76
  /** Custom BedrockRuntimeClient instance */
77
  client?: BedrockRuntimeClient;
78
  
79
  /** Configuration options for BedrockRuntimeClient */
80
  clientOptions?: BedrockRuntimeClientConfig;
81
  
82
  /** Enable streaming responses by default */
83
  streaming?: boolean;
84
  
85
  /** Model ID to use (default: "anthropic.claude-3-haiku-20240307-v1:0") */
86
  model?: string;
87
  
88
  /** AWS region for API calls */
89
  region?: string;
90
  
91
  /** AWS credentials for authentication */
92
  credentials?: CredentialType;
93
  
94
  /** Temperature for response randomness (0.0-1.0) */
95
  temperature?: number;
96
  
97
  /** Maximum tokens to generate */
98
  maxTokens?: number;
99
  
100
  /** Custom endpoint hostname override */
101
  endpointHost?: string;
102
  
103
  /** Top-p sampling parameter (0.0-1.0) */
104
  topP?: number;
105
  
106
  /** Additional model-specific request fields */
107
  additionalModelRequestFields?: __DocumentType;
108
  
109
  /** Include usage metadata in streaming responses */
110
  streamUsage?: boolean;
111
  
112
  /** Guardrail configuration for content filtering */
113
  guardrailConfig?: GuardrailConfiguration;
114
  
115
  /** Performance configuration for latency optimization */
116
  performanceConfig?: PerformanceConfiguration;
117
  
118
  /** Supported tool choice values for this model */
119
  supportsToolChoiceValues?: Array<"auto" | "any" | "tool">;
120
}
121
```
122

123
### Call Options
124

125
Runtime options that can be passed to model methods for per-request customization.
126

127
```typescript { .api }
128
interface ChatBedrockConverseCallOptions extends BaseChatModelCallOptions {
129
  /** Stop sequences to halt generation */
130
  stop?: string[];
131
  
132
  /** Tools available for function calling */
133
  tools?: ChatBedrockConverseToolType[];
134
  
135
  /** Tool choice strategy ("auto", "any", tool name, or BedrockToolChoice object) */
136
  tool_choice?: BedrockConverseToolChoice;
137
  
138
  /** Additional model-specific fields for this request */
139
  additionalModelRequestFields?: __DocumentType;
140
  
141
  /** Include usage metadata in streaming for this request */
142
  streamUsage?: boolean;
143
  
144
  /** Guardrail configuration for this request */
145
  guardrailConfig?: GuardrailConfiguration;
146
  
147
  /** Performance configuration for this request */
148
  performanceConfig?: PerformanceConfiguration;
149
}
150
```
151

152
### Function Calling
153

154
Comprehensive tool binding and function calling capabilities with intelligent tool choice selection.
155

156
```typescript { .api }
157
type ChatBedrockConverseToolType = BindToolsInput | BedrockTool;
158

159
type BedrockConverseToolChoice = "auto" | "any" | string | BedrockToolChoice;
160

161
type BedrockToolChoice = ToolChoice.AnyMember | ToolChoice.AutoMember | ToolChoice.ToolMember;
162
```
163

164
**Usage Examples:**
165

166
```typescript
167
import { z } from "zod";
168

169
// Define tools with Zod schemas
170
const weatherTool = {
171
  name: "get_weather",
172
  description: "Get current weather information",
173
  schema: z.object({
174
    location: z.string().describe("City and state, e.g. San Francisco, CA"),
175
    unit: z.enum(["celsius", "fahrenheit"]).optional()
176
  })
177
};
178

179
const calculatorTool = {
180
  name: "calculate",
181
  description: "Perform mathematical calculations",
182
  schema: z.object({
183
    expression: z.string().describe("Mathematical expression to evaluate")
184
  })
185
};
186

187
// Bind tools with automatic tool choice
188
const modelWithTools = model.bindTools([weatherTool, calculatorTool], {
189
  tool_choice: "auto" // Let model decide when to use tools
190
});
191

192
// Force tool usage
193
const modelForceWeather = model.bindTools([weatherTool], {
194
  tool_choice: "get_weather" // Always use weather tool
195
});
196

197
// Require any tool
198
const modelRequireTool = model.bindTools([weatherTool, calculatorTool], {
199
  tool_choice: "any" // Must use at least one tool
200
});
201

202
const result = await modelWithTools.invoke([
203
  new HumanMessage("What's the weather in Paris and what's 2 + 2?")
204
]);
205

206
// Access tool calls from response
207
if (result.tool_calls && result.tool_calls.length > 0) {
208
  result.tool_calls.forEach(call => {
209
    console.log(`Tool: ${call.name}, Args:`, call.args);
210
  });
211
}
212
```
213

214
### Structured Output
215

216
Force models to return structured JSON data using schema validation.
217

218
**Usage Examples:**
219

220
```typescript
221
import { z } from "zod";
222

223
// Define response structure
224
const JokeSchema = z.object({
225
  setup: z.string().describe("The setup of the joke"),
226
  punchline: z.string().describe("The punchline"),
227
  rating: z.number().min(1).max(10).optional().describe("Humor rating 1-10")
228
});
229

230
// Create structured output model
231
const structuredModel = model.withStructuredOutput(JokeSchema, {
232
  name: "generate_joke"
233
});
234

235
// Get structured response
236
const joke = await structuredModel.invoke([
237
  new HumanMessage("Tell me a joke about programming")
238
]);
239

240
console.log(joke); // { setup: "Why do programmers...", punchline: "...", rating: 8 }
241

242
// With raw response included
243
const modelWithRaw = model.withStructuredOutput(JokeSchema, {
244
  name: "generate_joke",
245
  includeRaw: true
246
});
247

248
const result = await modelWithRaw.invoke([
249
  new HumanMessage("Tell me a joke about cats")
250
]);
251

252
console.log(result.parsed); // Structured joke object
253
console.log(result.raw); // Original AIMessage
254
```
255

256
### Multimodal Input
257

258
Support for text, images, and documents in conversations.
259

260
**Usage Examples:**
261

262
```typescript
263
import { HumanMessage } from "@langchain/core/messages";
264

265
// Image analysis
266
const imageMessage = new HumanMessage({
267
  content: [
268
    { type: "text", text: "Describe what you see in this image" },
269
    {
270
      type: "image_url",
271
      image_url: {
272
        url: "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD..."
273
      }
274
    }
275
  ]
276
});
277

278
const imageResponse = await model.invoke([imageMessage]);
279

280
// Document analysis
281
const documentMessage = new HumanMessage({
282
  content: [
283
    { type: "text", text: "Summarize this document" },
284
    {
285
      type: "document",
286
      document: {
287
        format: "pdf",
288
        name: "report.pdf",
289
        source: {
290
          bytes: pdfBuffer
291
        }
292
      }
293
    }
294
  ]
295
});
296

297
const docResponse = await model.invoke([documentMessage]);
298
```
299

300
### Advanced Features
301

302
#### Reasoning Content Support
303

304
Claude models can produce reasoning blocks showing their thought process.
305

306
```typescript
307
// Reasoning content is automatically handled in streaming
308
const stream = await model.stream([
309
  new HumanMessage("Solve this step by step: What is 15% of 240?")
310
]);
311

312
for await (const chunk of stream) {
313
  // Reasoning content appears in response_metadata
314
  if (chunk.response_metadata?.reasoning) {
315
    console.log("Reasoning:", chunk.response_metadata.reasoning);
316
  }
317
  console.log("Content:", chunk.content);
318
}
319
```
320

321
#### Prompt Caching
322

323
Optimize costs and latency by caching frequently used prompts.
324

325
```typescript
326
import { SystemMessage } from "@langchain/core/messages";
327

328
const systemMessage = new SystemMessage({
329
  content: [
330
    { type: "text", text: "You are a helpful coding assistant with expertise in Python and JavaScript." },
331
    { cachePoint: { type: "default" } }, // Cache point marker
332
    { type: "text", text: "Always provide working code examples and explain your reasoning." }
333
  ]
334
});
335

336
const response = await model.invoke([
337
  systemMessage,
338
  new HumanMessage("How do I implement a binary search in Python?")
339
]);
340
```
341

342
#### Guardrails Integration
343

344
Built-in support for AWS Bedrock Guardrails for content filtering.
345

346
```typescript
347
const guardedModel = new ChatBedrockConverse({
348
  region: "us-east-1",
349
  model: "anthropic.claude-3-5-sonnet-20240620-v1:0",
350
  guardrailConfig: {
351
    guardrailIdentifier: "your-guardrail-id",
352
    guardrailVersion: "1",
353
    trace: "enabled"
354
  }
355
});
356

357
// Guardrails are automatically applied to requests and responses
358
const response = await guardedModel.invoke([
359
  new HumanMessage("Your message here")
360
]);
361
```
362

363
#### Performance Configuration
364

365
Optimize for reduced latency using performance settings.
366

367
```typescript
368
const optimizedModel = new ChatBedrockConverse({
369
  region: "us-east-1",
370
  model: "anthropic.claude-3-5-sonnet-20240620-v1:0",
371
  performanceConfig: {
372
    latency: "optimized" // or "standard"
373
  }
374
});
375
```