0
# Groq
1

2
The official Python library for the Groq API, providing access to Groq's high-performance language models and services. Groq enables fast inference for large language models with comprehensive support for chat completions, embeddings, audio processing, and more.
3

4
## Package Information
5

6
- **Package Name**: groq
7
- **Language**: Python
8
- **Installation**: `pip install groq`
9
- **Python Version**: >=3.8
10

11
## Core Imports
12

13
```python
14
import groq
15
```
16

17
Standard client usage:
18

19
```python
20
from groq import Groq
21
```
22

23
Async client usage:
24

25
```python
26
from groq import AsyncGroq
27
```
28

29
Type imports:
30

31
```python
32
from groq import types
33
```
34

35
## Basic Usage
36

37
```python
38
import os
39
from groq import Groq
40

41
# Initialize client with API key from environment
42
client = Groq(api_key=os.environ.get("GROQ_API_KEY"))
43

44
# Create a chat completion
45
completion = client.chat.completions.create(
46
    messages=[
47
        {"role": "system", "content": "You are a helpful assistant."},
48
        {"role": "user", "content": "Hello! How are you?"}
49
    ],
50
    model="llama3-8b-8192",
51
    max_tokens=100,
52
    temperature=0.7
53
)
54

55
print(completion.choices[0].message.content)
56

57
# Stream a chat completion
58
stream = client.chat.completions.create(
59
    messages=[
60
        {"role": "user", "content": "Write a short poem about Python"}
61
    ],
62
    model="llama3-8b-8192",
63
    stream=True
64
)
65

66
for chunk in stream:
67
    if chunk.choices[0].delta.content is not None:
68
        print(chunk.choices[0].delta.content, end="")
69
```
70

71
## Architecture
72

73
The Groq library follows a structured client-resource pattern:
74

75
- **Client Classes**: `Groq` (sync) and `AsyncGroq` (async) serve as entry points
76
- **Resource APIs**: Organized by functionality (chat, audio, embeddings, models, batches, files)
77
- **Streaming Support**: Real-time response streaming for compatible endpoints
78
- **Type Safety**: Comprehensive type definitions for all request parameters and response fields
79
- **Error Handling**: Structured exception hierarchy for different error scenarios
80
- **Response Wrappers**: Raw response access and streaming response handling
81

82
## Capabilities
83

84
### Chat Completions
85

86
High-performance chat completions with streaming support, function calling, tool usage, and advanced features like reasoning modes and search integration.
87

88
```python { .api }
89
def create(
90
    messages: Iterable[ChatCompletionMessageParam],
91
    model: str,
92
    max_tokens: Optional[int] = None,
93
    temperature: Optional[float] = None,
94
    stream: Optional[bool] = None,
95
    tools: Optional[Iterable[ChatCompletionToolParam]] = None,
96
    **kwargs
97
) -> ChatCompletion | Stream[ChatCompletionChunk]: ...
98
```
99

100
[Chat Completions](./chat-completions.md)
101

102
### Text Embeddings
103

104
Generate high-quality vector embeddings for text inputs, supporting both single strings and batch processing.
105

106
```python { .api }
107
def create(
108
    input: Union[str, List[str]],
109
    model: Union[str, Literal["nomic-embed-text-v1_5"]],
110
    **kwargs  
111
) -> CreateEmbeddingResponse: ...
112
```
113

114
[Text Embeddings](./embeddings.md)
115

116
### Audio Processing
117

118
Comprehensive audio capabilities including speech-to-text transcription, translation, and text-to-speech synthesis.
119

120
```python { .api }
121
# client.audio.transcriptions.create()
122
def create(model: str, file: FileTypes, **kwargs) -> Transcription: ...
123
# client.audio.translations.create()  
124
def create(model: str, file: FileTypes, **kwargs) -> Translation: ...
125
# client.audio.speech.create()
126
def create(input: str, model: str, voice: str, **kwargs) -> bytes: ...
127
```
128

129
[Audio Processing](./audio.md)
130

131
### Model Management
132

133
Access model information, list available models, and manage model lifecycle operations.
134

135
```python { .api }
136
def list(**kwargs) -> ModelListResponse: ...
137
def retrieve(model: str, **kwargs) -> Model: ...
138
def delete(model: str, **kwargs) -> ModelDeleted: ...
139
```
140

141
[Model Management](./models.md)
142

143
### File Operations
144

145
Upload, manage, and organize files for use with various Groq services and batch processing.
146

147
```python { .api }
148
def create(file: FileTypes, purpose: str, **kwargs) -> FileCreateResponse: ...
149
def list(**kwargs) -> FileListResponse: ...
150
def retrieve(file_id: str, **kwargs) -> FileInfoResponse: ...
151
def delete(file_id: str, **kwargs) -> FileDeleteResponse: ...
152
```
153

154
[File Operations](./files.md)
155

156
### Batch Processing
157

158
Submit and manage batch jobs for processing large volumes of requests efficiently.
159

160
```python { .api }
161
def create(input_file_id: str, endpoint: str, completion_window: str, **kwargs) -> BatchCreateResponse: ...
162
def retrieve(batch_id: str, **kwargs) -> BatchRetrieveResponse: ...
163
def cancel(batch_id: str, **kwargs) -> BatchCancelResponse: ...
164
def list(**kwargs) -> BatchListResponse: ...
165
```
166

167
[Batch Processing](./batches.md)
168

169
## Client Configuration
170

171
```python { .api }
172
class Groq:
173
    def __init__(
174
        self,
175
        api_key: str | None = None,
176
        base_url: str | httpx.URL | None = None,
177
        timeout: Union[float, Timeout, None, NotGiven] = NOT_GIVEN,
178
        max_retries: int = DEFAULT_MAX_RETRIES,
179
        default_headers: Mapping[str, str] | None = None,
180
        default_query: Mapping[str, object] | None = None,
181
        http_client: httpx.Client | None = None,
182
        _strict_response_validation: bool = False
183
    ): ...
184

185
class AsyncGroq:
186
    def __init__(
187
        self,
188
        api_key: str | None = None,
189
        base_url: str | httpx.URL | None = None,
190
        timeout: Union[float, Timeout, None, NotGiven] = NOT_GIVEN,
191
        max_retries: int = DEFAULT_MAX_RETRIES,
192
        default_headers: Mapping[str, str] | None = None,
193
        default_query: Mapping[str, object] | None = None,
194
        http_client: httpx.AsyncClient | None = None,
195
        _strict_response_validation: bool = False
196
    ): ...
197
```
198

199
## Error Handling
200

201
```python { .api }
202
class GroqError(Exception): ...
203
class APIError(GroqError): ...
204
class APIStatusError(APIError): ...
205
class APITimeoutError(APIConnectionError): ...
206
class APIConnectionError(APIError): ...
207
class APIResponseValidationError(APIError): ...
208
class BadRequestError(APIStatusError): ...
209
class AuthenticationError(APIStatusError): ...
210
class PermissionDeniedError(APIStatusError): ...
211
class NotFoundError(APIStatusError): ...
212
class ConflictError(APIStatusError): ...
213
class UnprocessableEntityError(APIStatusError): ...
214
class RateLimitError(APIStatusError): ...
215
class InternalServerError(APIStatusError): ...
216
```
217

218
## Core Types
219

220
```python { .api }
221
class BaseModel: ...
222
class NotGiven: ...
223
NOT_GIVEN: NotGiven
224
class Timeout: ...
225
class RequestOptions: ...
226

227
# Response wrappers
228
class APIResponse: ...
229
class AsyncAPIResponse: ...
230
class Stream: ...
231
class AsyncStream: ...
232

233
# HTTP clients
234
class DefaultHttpxClient: ...
235
class DefaultAsyncHttpxClient: ...
236
class DefaultAioHttpClient: ...
237
```
238

239
## Constants
240

241
```python { .api }
242
DEFAULT_TIMEOUT: httpx.Timeout
243
DEFAULT_MAX_RETRIES: int
244
DEFAULT_CONNECTION_LIMITS: httpx.Limits
245
```
246

247
## Utilities
248

249
```python { .api }
250
def file_from_path(path: str) -> FileTypes: ...
251
```