0
# Batch Processing
1

2
Submit and manage batch processing jobs for handling large volumes of requests efficiently. Batch processing is ideal for processing many requests asynchronously at lower cost.
3

4
## Capabilities
5

6
### Create Batch Job
7

8
Submit a batch processing job with multiple requests.
9

10
```python { .api }
11
def create(
12
    input_files: List[str],
13
    endpoint: str,
14
    completion_window: str,
15
    metadata: Optional[dict] = None,
16
    **kwargs
17
) -> BatchJobOut:
18
    """
19
    Create a batch processing job.
20

21
    Parameters:
22
    - input_files: List of file IDs containing batch requests
23
    - endpoint: API endpoint to process ("/v1/chat/completions", "/v1/embeddings")
24
    - completion_window: Time window for completion ("24h")
25
    - metadata: Optional metadata for the batch
26

27
    Returns:
28
    BatchJobOut with batch job details
29
    """
30

31
def list(
32
    limit: Optional[int] = None,
33
    after: Optional[str] = None,
34
    **kwargs
35
) -> BatchJobsOut:
36
    """
37
    List batch processing jobs.
38

39
    Parameters:
40
    - limit: Maximum number of jobs to return
41
    - after: Cursor for pagination
42

43
    Returns:
44
    BatchJobsOut with list of batch jobs
45
    """
46

47
def get(batch_id: str, **kwargs) -> BatchJobOut:
48
    """
49
    Retrieve a batch processing job.
50

51
    Parameters:
52
    - batch_id: Unique identifier of the batch job
53

54
    Returns:
55
    BatchJobOut with detailed batch information
56
    """
57

58
def cancel(batch_id: str, **kwargs) -> BatchJobOut:
59
    """
60
    Cancel a batch processing job.
61

62
    Parameters:
63
    - batch_id: Unique identifier of the batch to cancel
64

65
    Returns:
66
    Updated batch job with cancelled status
67
    """
68
```
69

70
## Usage Examples
71

72
### Prepare Batch Requests
73

74
```python
75
import json
76

77
# Create batch requests file
78
batch_requests = []
79

80
# Add multiple chat completion requests
81
for i in range(100):
82
    request = {
83
        "custom_id": f"request-{i}",
84
        "method": "POST",
85
        "url": "/v1/chat/completions",
86
        "body": {
87
            "model": "mistral-small-latest",
88
            "messages": [
89
                {"role": "user", "content": f"Summarize topic number {i}"}
90
            ],
91
            "max_tokens": 100
92
        }
93
    }
94
    batch_requests.append(request)
95

96
# Save to JSONL file
97
with open("batch_requests.jsonl", "w") as f:
98
    for request in batch_requests:
99
        f.write(json.dumps(request) + "\n")
100
```
101

102
### Submit Batch Job
103

104
```python
105
from mistralai import Mistral
106

107
client = Mistral(api_key="your-api-key")
108

109
# Upload batch requests file
110
upload_result = client.files.upload(
111
    file="batch_requests.jsonl",
112
    purpose="batch"
113
)
114

115
# Create batch job
116
batch_job = client.batch.create(
117
    input_files=[upload_result.id],
118
    endpoint="/v1/chat/completions",
119
    completion_window="24h",
120
    metadata={"description": "Batch summarization job"}
121
)
122

123
print(f"Created batch job: {batch_job.id}")
124
print(f"Status: {batch_job.status}")
125
```
126

127
### Monitor Batch Progress
128

129
```python
130
import time
131

132
# Monitor batch job
133
while True:
134
    batch_status = client.batch.get(batch_job.id)
135
    print(f"Batch status: {batch_status.status}")
136
    print(f"Completed: {batch_status.request_counts.completed}")
137
    print(f"Failed: {batch_status.request_counts.failed}")
138
    print(f"Total: {batch_status.request_counts.total}")
139
    
140
    if batch_status.status in ["completed", "failed", "cancelled"]:
141
        break
142
    
143
    time.sleep(60)  # Check every minute
144

145
# Download results
146
if batch_status.status == "completed" and batch_status.output_file_id:
147
    results = client.files.download(batch_status.output_file_id)
148
    
149
    # Save results
150
    with open("batch_results.jsonl", "wb") as f:
151
        f.write(results)
152
    
153
    print("Batch processing completed successfully!")
154
```
155

156
### Process Results
157

158
```python
159
import json
160

161
# Read and process batch results
162
with open("batch_results.jsonl", "r") as f:
163
    for line in f:
164
        result = json.loads(line)
165
        custom_id = result["custom_id"]
166
        
167
        if result["response"]["status_code"] == 200:
168
            response_body = result["response"]["body"]
169
            content = response_body["choices"][0]["message"]["content"]
170
            print(f"{custom_id}: {content[:100]}...")
171
        else:
172
            error = result["response"]["body"]
173
            print(f"{custom_id}: Error - {error}")
174
```
175

176
## Types
177

178
### Batch Job Types
179

180
```python { .api }
181
class BatchJobOut:
182
    id: str
183
    object: str
184
    endpoint: str
185
    errors: Optional[dict]
186
    input_file_id: str
187
    completion_window: str
188
    status: str
189
    output_file_id: Optional[str]
190
    error_file_id: Optional[str]
191
    created_at: int
192
    in_progress_at: Optional[int]
193
    expires_at: int
194
    finalizing_at: Optional[int]
195
    completed_at: Optional[int]
196
    failed_at: Optional[int]
197
    expired_at: Optional[int]
198
    cancelling_at: Optional[int]
199
    cancelled_at: Optional[int]
200
    request_counts: BatchJobRequestCounts
201
    metadata: Optional[dict]
202

203
class BatchJobsOut:
204
    object: str
205
    data: List[BatchJobOut]
206
    first_id: Optional[str]
207
    last_id: Optional[str]
208
    has_more: bool
209

210
class BatchJobRequestCounts:
211
    total: int
212
    completed: int
213
    failed: int
214
```
215

216
### Batch Input Format
217

218
```python { .api }
219
class BatchJobIn:
220
    input_files: List[str]
221
    endpoint: str
222
    completion_window: str
223
    metadata: Optional[dict]
224
```
225

226
### Status Values
227

228
- **validating**: Batch is being validated
229
- **in_progress**: Batch is being processed
230
- **finalizing**: Batch processing is finishing
231
- **completed**: Batch completed successfully
232
- **failed**: Batch failed to process
233
- **expired**: Batch expired before completion
234
- **cancelling**: Batch is being cancelled
235
- **cancelled**: Batch was cancelled
236

237
### Error Handling
238

239
```python { .api }
240
class BatchError:
241
    code: Optional[str]
242
    message: Optional[str]
243
    param: Optional[str]
244
    line: Optional[int]
245
```