0
# LangChain Groq
1

2
@langchain/groq provides a LangChain.js integration for Groq's high-performance language model inference API. It implements the ChatGroq class that extends LangChain's chat model interface to interact with Groq's API, supporting various models like llama-3.3-70b-versatile with built-in streaming, tool calling, and structured output capabilities.
3

4
## Package Information
5

6
- **Package Name**: @langchain/groq
7
- **Package Type**: npm
8
- **Language**: TypeScript
9
- **Installation**: `npm install @langchain/groq @langchain/core`
10

11
## Core Imports
12

13
```typescript
14
import { ChatGroq } from "@langchain/groq";
15
```
16

17
For CommonJS:
18

19
```javascript
20
const { ChatGroq } = require("@langchain/groq");
21
```
22

23
## Basic Usage
24

25
```typescript
26
import { ChatGroq } from "@langchain/groq";
27
import { HumanMessage } from "@langchain/core/messages";
28

29
// Create a chat model instance
30
const model = new ChatGroq({
31
  apiKey: process.env.GROQ_API_KEY, // or provide directly
32
  model: "llama-3.3-70b-versatile",
33
  temperature: 0.7,
34
});
35

36
// Basic text generation
37
const message = new HumanMessage("What color is the sky?");
38
const response = await model.invoke([message]);
39
console.log(response.content);
40

41
// Streaming responses
42
for await (const chunk of await model.stream([message])) {
43
  console.log(chunk.content);
44
}
45
```
46

47
## Architecture
48

49
The @langchain/groq package is built around these key components:
50

51
- **ChatGroq Class**: Main chat model implementation extending LangChain's BaseChatModel
52
- **Groq SDK Integration**: Uses the official groq-sdk for API communication
53
- **Message Conversion**: Handles translation between LangChain and Groq message formats
54
- **Streaming Support**: Native streaming with token usage reporting
55
- **Tool Integration**: Support for function calling via LangChain's tool binding system
56
- **Structured Output**: JSON schema-based structured response generation
57

58
## Capabilities
59

60
### Chat Model
61

62
Core chat model functionality for generating responses from Groq's language models.
63

64
```typescript { .api }
65
class ChatGroq extends BaseChatModel<ChatGroqCallOptions, AIMessageChunk> {
66
  constructor(fields: ChatGroqInput);
67
  
68
  invoke(
69
    messages: BaseMessage[],
70
    options?: Partial<ChatGroqCallOptions>
71
  ): Promise<BaseMessage>;
72
  
73
  stream(
74
    messages: BaseMessage[],
75
    options?: Partial<ChatGroqCallOptions>
76
  ): Promise<AsyncIterable<BaseMessageChunk>>;
77
  
78
  bindTools(
79
    tools: ChatGroqToolType[],
80
    kwargs?: Partial<ChatGroqCallOptions>
81
  ): Runnable<BaseLanguageModelInput, AIMessageChunk, ChatGroqCallOptions>;
82
  
83
  withStructuredOutput<RunOutput extends Record<string, any> = Record<string, any>>(
84
    outputSchema: InteropZodType<RunOutput> | Record<string, any>,
85
    config?: StructuredOutputMethodOptions<boolean>
86
  ): Runnable<BaseLanguageModelInput, RunOutput> | Runnable<BaseLanguageModelInput, { raw: BaseMessage; parsed: RunOutput }>;
87
}
88
```
89

90
**Usage Examples:**
91

92
```typescript
93
import { ChatGroq } from "@langchain/groq";
94
import { HumanMessage, SystemMessage } from "@langchain/core/messages";
95

96
// Initialize with configuration
97
const chat = new ChatGroq({
98
  model: "llama-3.3-70b-versatile",
99
  temperature: 0.7,
100
  maxTokens: 1000,
101
  apiKey: "your-api-key",
102
});
103

104
// Multi-turn conversation
105
const messages = [
106
  new SystemMessage("You are a helpful assistant."),
107
  new HumanMessage("Explain quantum computing in simple terms."),
108
];
109

110
const response = await chat.invoke(messages);
111

112
// With runtime options
113
const configuredResponse = await chat.invoke(messages, {
114
  temperature: 0.2,
115
  stop: ["END"],
116
  headers: { "Custom-Header": "value" },
117
});
118
```
119

120
### Tool Calling
121

122
Bind tools to enable function calling capabilities with Groq models that support it.
123

124
```typescript { .api }
125
bindTools(
126
  tools: ChatGroqToolType[],
127
  kwargs?: Partial<ChatGroqCallOptions>
128
): Runnable<BaseLanguageModelInput, AIMessageChunk, ChatGroqCallOptions>;
129

130
type ChatGroqToolType = BindToolsInput | ChatCompletionTool;
131
```
132

133
**Usage Examples:**
134

135
```typescript
136
import { ChatGroq } from "@langchain/groq";
137
import { tool } from "@langchain/core/tools";
138
import { z } from "zod";
139

140
// Define a tool
141
const weatherTool = tool(
142
  async ({ location }: { location: string }) => {
143
    return `The weather in ${location} is sunny and 72°F`;
144
  },
145
  {
146
    name: "get_weather",
147
    description: "Get the current weather for a location",
148
    schema: z.object({
149
      location: z.string().describe("The city and state, e.g. San Francisco, CA"),
150
    }),
151
  }
152
);
153

154
// Bind tools to model
155
const modelWithTools = new ChatGroq({
156
  model: "llama3-groq-70b-8192-tool-use-preview",
157
}).bindTools([weatherTool]);
158

159
const response = await modelWithTools.invoke([
160
  new HumanMessage("What's the weather like in New York?")
161
]);
162

163
// Check for tool calls in response
164
if (response.tool_calls?.length > 0) {
165
  console.log("Tool calls:", response.tool_calls);
166
}
167
```
168

169
### Structured Output
170

171
Generate structured JSON responses using Zod schemas or JSON schemas.
172

173
```typescript { .api }
174
withStructuredOutput<RunOutput extends Record<string, any> = Record<string, any>>(
175
  outputSchema: InteropZodType<RunOutput> | Record<string, any>,
176
  config?: StructuredOutputMethodOptions<boolean>
177
): Runnable<BaseLanguageModelInput, RunOutput> | Runnable<BaseLanguageModelInput, { raw: BaseMessage; parsed: RunOutput }>;
178

179
interface StructuredOutputMethodOptions<IncludeRaw extends boolean = false> {
180
  name?: string;
181
  method?: "functionCalling" | "jsonMode";
182
  includeRaw?: IncludeRaw;
183
}
184
```
185

186
**Usage Examples:**
187

188
```typescript
189
import { ChatGroq } from "@langchain/groq";
190
import { z } from "zod";
191

192
// Define output schema
193
const PersonSchema = z.object({
194
  name: z.string().describe("Person's full name"),
195
  age: z.number().describe("Person's age in years"),
196
  occupation: z.string().describe("Person's job or profession"),
197
  location: z.string().describe("City and country where person lives"),
198
});
199

200
// Create structured output model
201
const structuredModel = new ChatGroq({
202
  model: "llama3-groq-70b-8192-tool-use-preview",
203
}).withStructuredOutput(PersonSchema, { name: "extract_person_info" });
204

205
// Generate structured response
206
const result = await structuredModel.invoke([
207
  new HumanMessage("Tell me about Alice Johnson, a 32-year-old software engineer from Toronto, Canada.")
208
]);
209

210
console.log(result);
211
// Output: { name: "Alice Johnson", age: 32, occupation: "software engineer", location: "Toronto, Canada" }
212

213
// Include raw response
214
const modelWithRaw = new ChatGroq({
215
  model: "llama3-groq-70b-8192-tool-use-preview",
216
}).withStructuredOutput(PersonSchema, { includeRaw: true });
217

218
const resultWithRaw = await modelWithRaw.invoke([...]);
219
console.log(resultWithRaw.parsed); // Structured data
220
console.log(resultWithRaw.raw);    // Original message
221
```
222

223
### Streaming
224

225
Stream response chunks for real-time applications.
226

227
```typescript { .api }
228
stream(
229
  messages: BaseMessage[],
230
  options?: Partial<ChatGroqCallOptions>
231
): Promise<AsyncIterable<BaseMessageChunk>>;
232
```
233

234
**Usage Examples:**
235

236
```typescript
237
import { ChatGroq } from "@langchain/groq";
238
import { HumanMessage } from "@langchain/core/messages";
239
import { concat } from "@langchain/core/utils/stream";
240

241
const chat = new ChatGroq({
242
  model: "llama-3.3-70b-versatile",
243
  streamUsage: true, // Include token usage in stream
244
});
245

246
// Basic streaming
247
for await (const chunk of await chat.stream([
248
  new HumanMessage("Write a short story about a robot learning to paint.")
249
])) {
250
  console.log(chunk.content);
251
}
252

253
// Aggregate streaming chunks
254
const stream = await chat.stream([new HumanMessage("Explain photosynthesis")]);
255
let full: AIMessageChunk | undefined;
256
for await (const chunk of stream) {
257
  full = !full ? chunk : concat(full, chunk);
258
}
259
console.log(full.content);
260

261
// Stream with options
262
for await (const chunk of await chat.stream([...], {
263
  temperature: 0.3,
264
  max_tokens: 500,
265
  stream_options: { include_usage: true },
266
})) {
267
  if (chunk.response_metadata?.usage) {
268
    console.log("Token usage:", chunk.response_metadata.usage);
269
  }
270
}
271
```
272

273
## Configuration Types
274

275
### ChatGroqInput
276

277
Constructor configuration interface for ChatGroq.
278

279
```typescript { .api }
280
interface ChatGroqInput extends BaseChatModelParams {
281
  /** Required: The Groq model name to use */
282
  model: string;
283
  
284
  /** API key (defaults to GROQ_API_KEY environment variable) */
285
  apiKey?: string;
286
  
287
  /** Sampling temperature between 0 and 2 (default: 0.7) */
288
  temperature?: number;
289
  
290
  /** Maximum number of tokens in response */
291
  maxTokens?: number;
292
  
293
  /** Nucleus sampling parameter (0-1) */
294
  topP?: number;
295
  
296
  /** Frequency penalty for reducing repetition (-2 to 2) */
297
  frequencyPenalty?: number;
298
  
299
  /** Presence penalty for encouraging topic diversity (-2 to 2) */
300
  presencePenalty?: number;
301
  
302
  /** Number of completions to generate */
303
  n?: number;
304
  
305
  /** Token probability adjustments */
306
  logitBias?: Record<string, number>;
307
  
308
  /** End-user identifier for monitoring */
309
  user?: string;
310
  
311
  /** Enable streaming responses */
312
  streaming?: boolean;
313
  
314
  /** Include token usage data in streamed chunks (default: true) */
315
  streamUsage?: boolean;
316
  
317
  /** Return log probabilities of output tokens */
318
  logprobs?: boolean;
319
  
320
  /** Number of most likely tokens to return at each position (0-5) */
321
  topLogprobs?: number;
322
  
323
  /** Stop sequences (up to 4) */
324
  stop?: string | null | Array<string>;
325
  stopSequences?: Array<string>;
326
  
327
  /** Override API base URL */
328
  baseUrl?: string;
329
  
330
  /** Request timeout in milliseconds */
331
  timeout?: number;
332
  
333
  /** Custom HTTP agent */
334
  httpAgent?: any;
335
  
336
  /** Custom fetch function */
337
  fetch?: (...args: any) => any;
338
  
339
  /** Default headers for all requests */
340
  defaultHeaders?: Record<string, string>;
341
  
342
  /** Default query parameters for all requests */
343
  defaultQuery?: Record<string, string>;
344
}
345
```
346

347
### ChatGroqCallOptions
348

349
Runtime options for invoke/stream methods.
350

351
```typescript { .api }
352
interface ChatGroqCallOptions extends BaseChatModelCallOptions {
353
  /** Additional headers for this specific request */
354
  headers?: Record<string, string | null | undefined>;
355
  
356
  /** Index of the prompt in a list of prompts */
357
  promptIndex?: number;
358
  
359
  /** Stream configuration options */
360
  stream_options?: {
361
    /** Whether to include token usage in the stream */
362
    include_usage: boolean;
363
  };
364
  
365
  /** Tools for function calling */
366
  tools?: ChatGroqToolType[];
367
  
368
  /** Override temperature for this call */
369
  temperature?: number;
370
  
371
  /** Override max tokens for this call */
372
  max_tokens?: number;
373
  max_completion_tokens?: number;
374
  
375
  /** Override stop sequences for this call */
376
  stop?: string | string[];
377
  
378
  /** Tool choice strategy */
379
  tool_choice?: string | {
380
    type: "function";
381
    function: { name: string };
382
  };
383
  
384
  /** Response format specification */
385
  response_format?: {
386
    type: "json_object" | "text";
387
  };
388
  
389
  /** Seed for deterministic sampling */
390
  seed?: number;
391
  
392
  /** Enable parallel tool calls */
393
  parallel_tool_calls?: boolean;
394
  
395
  /** Frequency penalty override */
396
  frequency_penalty?: number;
397
  
398
  /** Presence penalty override */
399
  presence_penalty?: number;
400
  
401
  /** Logit bias override */
402
  logit_bias?: Record<string, number>;
403
  
404
  /** Log probabilities override */
405
  logprobs?: boolean;
406
  
407
  /** Top log probabilities override */
408
  top_logprobs?: number;
409
  
410
  /** Top-p override */
411
  top_p?: number;
412
  
413
  /** Reasoning format for compatible models */
414
  reasoning_format?: "text";
415
  
416
  /** Service tier specification */
417
  service_tier?: "auto" | "default";
418
}
419
```
420

421
### Response Types
422

423
Types returned by ChatGroq methods.
424

425
```typescript { .api }
426
interface TimingMetadata {
427
  /** Time spent generating tokens (seconds) */
428
  completion_time?: number;
429
  
430
  /** Time spent processing input tokens (seconds) */
431
  prompt_time?: number;
432
  
433
  /** Time request spent in queue (seconds) */
434
  queue_time?: number;
435
  
436
  /** Total time (completion_time + prompt_time) (seconds) */
437
  total_time?: number;
438
}
439

440
interface TokenUsage {
441
  /** Number of tokens in completion */
442
  completionTokens?: number;
443
  
444
  /** Number of tokens in prompt */
445
  promptTokens?: number;
446
  
447
  /** Total tokens used */
448
  totalTokens?: number;
449
}
450
```
451

452
## Utility Functions
453

454
### Message Conversion
455

456
Helper functions for converting between LangChain and Groq message formats.
457

458
```typescript { .api }
459
/**
460
 * Convert LangChain message type to Groq role string
461
 */
462
function messageToGroqRole(message: BaseMessage): GroqRoleEnum;
463

464
type GroqRoleEnum = "system" | "assistant" | "user" | "function";
465
```
466

467
## Error Handling
468

469
The ChatGroq class throws standard JavaScript errors for various conditions:
470

471
- **Authentication Error**: When API key is missing or invalid
472
- **Rate Limit Error**: When API rate limits are exceeded  
473
- **Model Error**: When specified model is not available
474
- **Validation Error**: When request parameters are invalid
475
- **Network Error**: When network connectivity issues occur
476
- **Timeout Error**: When requests exceed the specified timeout
477

478
Common error handling pattern:
479

480
```typescript
481
try {
482
  const response = await chat.invoke([message]);
483
  console.log(response.content);
484
} catch (error) {
485
  if (error.message.includes("API key")) {
486
    console.error("Authentication failed - check your API key");
487
  } else if (error.message.includes("rate limit")) {
488
    console.error("Rate limit exceeded - implement retry logic");
489
  } else {
490
    console.error("Unexpected error:", error.message);
491
  }
492
}
493
```
494

495
## Environment Variables
496

497
- **GROQ_API_KEY**: Your Groq API key (required if not provided in constructor)
498

499
```bash
500
export GROQ_API_KEY="your-groq-api-key-here"
501
```