0
# Together Python API Library
1

2
The Together Python API Library is the official Python client for Together's AI platform, providing comprehensive access to state-of-the-art AI models through synchronous and asynchronous interfaces. It enables developers to integrate chat completions, text completions, image generation, embeddings, reranking, audio processing, batch inference, and model fine-tuning capabilities into their Python 3.10+ applications.
3

4
## Package Information
5

6
- **Package Name**: together
7
- **Language**: Python
8
- **Installation**: `pip install together`
9
- **Documentation**: https://docs.together.ai/
10

11
## Core Imports
12

13
```python
14
from together import Together, AsyncTogether
15
```
16

17
For legacy compatibility:
18

19
```python
20
from together import Complete, AsyncComplete, Completion
21
from together import Embeddings, Files, Finetune, Image, Models
22
```
23

24
Import specific components:
25

26
```python
27
from together import resources, types, error
28
from together.types import ChatCompletionRequest, CompletionResponse
29
```
30

31
## Basic Usage
32

33
```python
34
from together import Together
35

36
# Initialize client (API key from TOGETHER_API_KEY env var or pass directly)
37
client = Together(api_key="your-api-key")
38

39
# Chat completion with text
40
response = client.chat.completions.create(
41
    model="meta-llama/Llama-3.2-3B-Instruct-Turbo",
42
    messages=[{"role": "user", "content": "Explain machine learning"}],
43
    max_tokens=500,
44
    temperature=0.7
45
)
46
print(response.choices[0].message.content)
47

48
# Text completion
49
response = client.completions.create(
50
    model="codellama/CodeLlama-34b-Python-hf",
51
    prompt="def fibonacci(n):",
52
    max_tokens=200,
53
    temperature=0.2
54
)
55
print(response.choices[0].text)
56

57
# Generate embeddings
58
response = client.embeddings.create(
59
    model="togethercomputer/m2-bert-80M-8k-retrieval",
60
    input=["Machine learning is amazing", "AI will transform everything"]
61
)
62
embeddings = [data.embedding for data in response.data]
63

64
# Generate image
65
response = client.images.generate(
66
    prompt="futuristic cityscape at sunset",
67
    model="stabilityai/stable-diffusion-xl-base-1.0",
68
    n=1,
69
    steps=20
70
)
71
image_data = response.data[0].b64_json
72
```
73

74
## Architecture
75

76
The Together library follows a resource-based architecture where each AI capability is organized into separate resource classes:
77

78
- **Client Classes**: `Together` (sync) and `AsyncTogether` (async) serve as the main entry points
79
- **Resource Classes**: Each API endpoint area (chat, completions, embeddings, etc.) has dedicated resource classes
80
- **Type System**: Comprehensive type definitions for all requests, responses, and data structures
81
- **Streaming Support**: Native streaming capabilities for real-time response processing
82
- **Error Handling**: Structured exception hierarchy for robust error management
83
- **Legacy API**: Backward compatibility with deprecated interfaces
84

85
The dual sync/async design enables flexible integration patterns, from simple synchronous scripts to high-performance asynchronous applications handling concurrent requests.
86

87
## Capabilities
88

89
### Chat Completions
90

91
Advanced conversational AI with support for multi-modal inputs including text, images, and video. Includes streaming, async operations, and comprehensive configuration options.
92

93
```python { .api }
94
def create(
95
    model: str,
96
    messages: List[dict],
97
    max_tokens: Optional[int] = None,
98
    temperature: Optional[float] = None,
99
    stream: bool = False,
100
    **kwargs
101
) -> ChatCompletionResponse: ...
102
```
103

104
[Chat Completions](./chat-completions.md)
105

106
### Text Completions
107

108
Raw text completion for code generation, creative writing, and general text completion tasks with streaming and batch processing support.
109

110
```python { .api }
111
def create(
112
    model: str,
113
    prompt: str,
114
    max_tokens: Optional[int] = None,
115
    temperature: Optional[float] = None,
116
    stream: bool = False,
117
    **kwargs
118
) -> CompletionResponse: ...
119
```
120

121
[Text Completions](./completions.md)
122

123
### Embeddings
124

125
High-dimensional vector representations of text for semantic search, clustering, classification, and similarity analysis with various embedding models.
126

127
```python { .api }
128
def create(
129
    model: str,
130
    input: Union[str, List[str]],
131
    **kwargs
132
) -> EmbeddingResponse: ...
133
```
134

135
[Embeddings](./embeddings.md)
136

137
### Image Generation
138

139
AI-powered image synthesis from text prompts with support for different models, resolutions, and generation parameters.
140

141
```python { .api }
142
def generate(
143
    prompt: str,
144
    model: str,
145
    n: int = 1,
146
    steps: Optional[int] = None,
147
    width: Optional[int] = None,
148
    height: Optional[int] = None,
149
    **kwargs
150
) -> ImageResponse: ...
151
```
152

153
[Image Generation](./images.md)
154

155
### File Management
156

157
File upload, listing, retrieval, and deletion operations for fine-tuning datasets and batch processing workflows.
158

159
```python { .api }
160
def upload(file: str, purpose: Optional[str] = None) -> FileResponse: ...
161
def list() -> FileList: ...
162
def retrieve(id: str) -> FileObject: ...
163
def retrieve_content(id: str) -> str: ...
164
def delete(id: str) -> FileDeleteResponse: ...
165
```
166

167
[File Management](./files.md)
168

169
### Models
170

171
Discovery and information retrieval for available AI models across different categories and capabilities.
172

173
```python { .api }
174
def list() -> List[ModelObject]: ...
175
```
176

177
[Models](./models.md)
178

179
### Fine-tuning
180

181
Custom model training with supervised fine-tuning and direct preference optimization, including job management and model downloading.
182

183
```python { .api }
184
def create(
185
    training_file: str,
186
    model: str,
187
    n_epochs: Optional[int] = None,
188
    batch_size: Optional[Union[str, int]] = None,
189
    learning_rate: Optional[float] = None,
190
    **kwargs
191
) -> FinetuneResponse: ...
192
```
193

194
[Fine-tuning](./fine-tuning.md)
195

196
### Reranking
197

198
Document relevance scoring and reordering for improved search and retrieval results with specialized reranking models.
199

200
```python { .api }
201
def create(
202
    model: str,
203
    query: str,
204
    documents: List[str],
205
    top_n: Optional[int] = None,
206
    **kwargs
207
) -> RerankResponse: ...
208
```
209

210
[Reranking](./rerank.md)
211

212
### Audio Processing
213

214
Speech synthesis, transcription, and translation capabilities supporting multiple languages and audio formats.
215

216
```python { .api }
217
# Speech synthesis
218
def create(
219
    model: str,
220
    input: str,
221
    voice: str,
222
    response_format: Optional[str] = None,
223
    **kwargs
224
) -> bytes: ...
225

226
# Audio transcription
227
def create(
228
    file: str,
229
    model: str,
230
    language: Optional[str] = None,
231
    **kwargs
232
) -> AudioTranscriptionResponse: ...
233
```
234

235
[Audio Processing](./audio.md)
236

237
### Batch Processing
238

239
Large-scale inference jobs with 24-hour turnaround time for processing thousands of requests efficiently and cost-effectively.
240

241
```python { .api }
242
def create_batch(
243
    file_id: str,
244
    endpoint: str,
245
    **kwargs
246
) -> BatchJob: ...
247
def get_batch(id: str) -> BatchJob: ...
248
def list_batches() -> List[BatchJob]: ...
249
```
250

251
[Batch Processing](./batch.md)
252

253
### Evaluation
254

255
Model performance evaluation with standardized metrics and comparison capabilities for assessing AI model quality and capabilities.
256

257
```python { .api }
258
def create(
259
    model: str,
260
    evaluation_type: str,
261
    dataset: str,
262
    **kwargs
263
) -> EvaluationCreateResponse: ...
264
```
265

266
[Evaluation](./evaluation.md)
267

268
### Dedicated Endpoint Management
269

270
Infrastructure management for deploying and scaling AI models on dedicated compute resources with autoscaling, hardware optimization, and performance monitoring.
271

272
```python { .api }
273
def create(
274
    *,
275
    model: str,
276
    hardware: str,
277
    min_replicas: int,
278
    max_replicas: int,
279
    display_name: Optional[str] = None,
280
    disable_prompt_cache: bool = False,
281
    disable_speculative_decoding: bool = False,
282
    state: Literal["STARTED", "STOPPED"] = "STARTED",
283
    inactive_timeout: Optional[int] = None
284
) -> DedicatedEndpoint: ...
285

286
def list(type: Optional[Literal["dedicated", "serverless"]] = None) -> List[ListEndpoint]: ...
287
def get(endpoint_id: str) -> DedicatedEndpoint: ...
288
def update(endpoint_id: str, **kwargs) -> DedicatedEndpoint: ...
289
def delete(endpoint_id: str) -> None: ...
290
def list_hardware(model: Optional[str] = None) -> List[HardwareWithStatus]: ...
291
```
292

293
[Dedicated Endpoint Management](./endpoints.md)
294

295
### Code Interpreter
296

297
Interactive code execution environment for running Python scripts with file upload support, session persistence, and comprehensive output capture.
298

299
```python { .api }
300
def run(
301
    code: str,
302
    language: Literal["python"],
303
    session_id: Optional[str] = None,
304
    files: Optional[List[Dict[str, Any]]] = None
305
) -> ExecuteResponse: ...
306
```
307

308
[Code Interpreter](./code-interpreter.md)
309

310
## Asynchronous Usage
311

312
All capabilities support asynchronous operations through the `AsyncTogether` client:
313

314
```python { .api }
315
class AsyncTogether:
316
    completions: AsyncCompletions
317
    chat: AsyncChat
318
    embeddings: AsyncEmbeddings
319
    files: AsyncFiles
320
    images: AsyncImages
321
    models: AsyncModels
322
    fine_tuning: AsyncFineTuning
323
    rerank: AsyncRerank
324
    audio: AsyncAudio
325
    batches: AsyncBatches
326
    evaluation: AsyncEvaluation
327
    code_interpreter: CodeInterpreter
328
```
329

330
## Core Types
331

332
### Client Configuration
333

334
```python { .api }
335
class Together:
336
    def __init__(
337
        self,
338
        api_key: Optional[str] = None,
339
        base_url: Optional[str] = None,
340
        timeout: Optional[float] = None, 
341
        max_retries: Optional[int] = None,
342
        supplied_headers: Optional[Dict[str, str]] = None
343
    ): ...
344

345
class AsyncTogether:
346
    def __init__(
347
        self,
348
        api_key: Optional[str] = None,
349
        base_url: Optional[str] = None,
350
        timeout: Optional[float] = None,
351
        max_retries: Optional[int] = None, 
352
        supplied_headers: Optional[Dict[str, str]] = None
353
    ): ...
354
```
355

356
### Request Base Types
357

358
```python { .api }
359
class TogetherRequest:
360
    """Base request type for Together API operations"""
361
    pass
362

363
class TogetherClient:
364
    """Core HTTP client for API communication"""
365
    def __init__(
366
        self,
367
        api_key: str,
368
        base_url: str,
369
        timeout: float,
370
        max_retries: int,
371
        supplied_headers: Optional[Dict[str, str]] = None
372
    ): ...
373
```
374

375
## CLI Interface
376

377
The library includes a comprehensive command-line interface accessible via the `together` command:
378

379
```bash
380
# Chat completions
381
together chat.completions \
382
  --message "user" "Explain quantum computing" \
383
  --model "meta-llama/Llama-3.2-3B-Instruct-Turbo"
384

385
# Text completions  
386
together completions \
387
  "def merge_sort(arr):" \
388
  --model "codellama/CodeLlama-34b-Python-hf" \
389
  --max-tokens 200
390

391
# Image generation
392
together images generate \
393
  "abstract art with vibrant colors" \
394
  --model "stabilityai/stable-diffusion-xl-base-1.0" \
395
  --n 2
396

397
# File operations
398
together files upload dataset.jsonl
399
together files list
400

401
# Model information
402
together models list
403
```
404

405
## Error Handling
406

407
The library provides structured error handling through the `together.error` module:
408

409
```python { .api }
410
class AuthenticationError(Exception):
411
    """Raised when API key is missing or invalid"""
412
    pass
413

414
class APIError(Exception):
415
    """Base class for API-related errors"""
416
    pass
417

418
class RateLimitError(APIError):
419
    """Raised when rate limits are exceeded"""
420
    pass
421
```
422

423
## Legacy API
424

425
Deprecated classes are available for backward compatibility:
426

427
```python { .api }
428
class Complete:
429
    """Legacy completion interface (deprecated)"""
430
    pass
431

432
class AsyncComplete:
433
    """Legacy async completion interface (deprecated)"""
434
    pass
435

436
class Completion:
437
    """Legacy completion result class (deprecated)"""
438
    pass
439
```