0
# Fine-tuning
1

2
Create and manage fine-tuning jobs to customize models for specific use cases. Fine-tuning allows you to create specialized models by training on your own data while building on Mistral's foundation models.
3

4
## Capabilities
5

6
### Create Fine-tuning Job
7

8
Start a new fine-tuning job with training data and configuration.
9

10
```python { .api }
11
def create(
12
    model: str,
13
    training_files: List[TrainingFile],
14
    validation_files: Optional[List[TrainingFile]] = None,
15
    hyperparameters: Optional[dict] = None,
16
    suffix: Optional[str] = None,
17
    integrations: Optional[List[dict]] = None,
18
    repositories: Optional[List[dict]] = None,
19
    **kwargs
20
) -> CompletionDetailedJobOut:
21
    """
22
    Create a fine-tuning job.
23

24
    Parameters:
25
    - model: Base model to fine-tune (e.g., "mistral-small-latest")
26
    - training_files: List of training data files
27
    - validation_files: Optional validation data files
28
    - hyperparameters: Training hyperparameters
29
    - suffix: Optional suffix for the fine-tuned model name
30
    - integrations: External integrations (e.g., W&B)
31
    - repositories: Repository configurations
32

33
    Returns:
34
    CompletionDetailedJobOut with job details and status
35
    """
36

37
def list(
38
    page: Optional[int] = None,
39
    page_size: Optional[int] = None,
40
    model: Optional[str] = None,
41
    created_after: Optional[int] = None,
42
    created_by_me: Optional[bool] = None,
43
    status: Optional[str] = None,
44
    wandb_project: Optional[str] = None,
45
    wandb_name: Optional[str] = None,
46
    suffix: Optional[str] = None,
47
    **kwargs
48
) -> JobsOut:
49
    """
50
    List fine-tuning jobs.
51

52
    Parameters:
53
    - page: Page number for pagination
54
    - page_size: Number of jobs per page
55
    - model: Filter by base model
56
    - created_after: Filter by creation timestamp
57
    - created_by_me: Filter by current user's jobs
58
    - status: Filter by job status
59
    - wandb_project: Filter by W&B project
60
    - wandb_name: Filter by W&B run name
61
    - suffix: Filter by model suffix
62

63
    Returns:
64
    JobsOut with list of fine-tuning jobs
65
    """
66

67
def get(job_id: str, **kwargs) -> CompletionDetailedJobOut:
68
    """
69
    Retrieve a fine-tuning job.
70

71
    Parameters:
72
    - job_id: Unique identifier of the fine-tuning job
73

74
    Returns:
75
    CompletionDetailedJobOut with detailed job information
76
    """
77

78
def cancel(job_id: str, **kwargs) -> CompletionDetailedJobOut:
79
    """
80
    Cancel a fine-tuning job.
81

82
    Parameters:
83
    - job_id: Unique identifier of the job to cancel
84

85
    Returns:
86
    Updated job information with cancelled status
87
    """
88

89
def start(job_id: str, **kwargs) -> CompletionDetailedJobOut:
90
    """
91
    Start a created fine-tuning job.
92

93
    Parameters:
94
    - job_id: Unique identifier of the job to start
95

96
    Returns:
97
    Updated job information with running status
98
    """
99
```
100

101
### Model Management
102

103
Manage fine-tuned models including archiving and updating.
104

105
```python { .api }
106
def archive_model(model_id: str, **kwargs) -> ArchiveFTModelOut:
107
    """
108
    Archive a fine-tuned model.
109

110
    Parameters:
111
    - model_id: ID of the fine-tuned model to archive
112

113
    Returns:
114
    Archive confirmation with model details
115
    """
116

117
def unarchive_model(model_id: str, **kwargs) -> UnarchiveFTModelOut:
118
    """
119
    Unarchive a fine-tuned model.
120

121
    Parameters:
122
    - model_id: ID of the archived model to restore
123

124
    Returns:
125
    Unarchive confirmation with model details
126
    """
127

128
def update_model(
129
    model_id: str,
130
    name: Optional[str] = None,
131
    description: Optional[str] = None,
132
    **kwargs
133
) -> CompletionFTModelOut:
134
    """
135
    Update fine-tuned model metadata.
136

137
    Parameters:
138
    - model_id: ID of the model to update
139
    - name: New name for the model
140
    - description: New description
141

142
    Returns:
143
    Updated model information
144
    """
145
```
146

147
## Usage Examples
148

149
### Prepare Training Data
150

151
```python
152
import json
153

154
# Create training data in JSONL format
155
training_data = [
156
    {
157
        "messages": [
158
            {"role": "user", "content": "What is the capital of France?"},
159
            {"role": "assistant", "content": "The capital of France is Paris."}
160
        ]
161
    },
162
    {
163
        "messages": [
164
            {"role": "user", "content": "Explain photosynthesis."},
165
            {"role": "assistant", "content": "Photosynthesis is the process by which plants convert sunlight, carbon dioxide, and water into glucose and oxygen."}
166
        ]
167
    }
168
]
169

170
# Save to JSONL file
171
with open("training_data.jsonl", "w") as f:
172
    for item in training_data:
173
        f.write(json.dumps(item) + "\n")
174
```
175

176
### Upload and Start Fine-tuning
177

178
```python
179
from mistralai import Mistral
180
from mistralai.models import TrainingFile
181

182
client = Mistral(api_key="your-api-key")
183

184
# Upload training file
185
upload_result = client.files.upload(
186
    file="training_data.jsonl",
187
    purpose="fine-tune"
188
)
189

190
# Create fine-tuning job
191
training_file = TrainingFile(
192
    file_id=upload_result.id,
193
    weight=1.0
194
)
195

196
job = client.fine_tuning.create(
197
    model="mistral-small-latest",
198
    training_files=[training_file],
199
    hyperparameters={
200
        "training_steps": 100,
201
        "learning_rate": 0.0001,
202
        "weight_decay": 0.1
203
    },
204
    suffix="my-custom-model"
205
)
206

207
print(f"Created fine-tuning job: {job.id}")
208
print(f"Status: {job.status}")
209
```
210

211
### Monitor Fine-tuning Progress
212

213
```python
214
import time
215

216
# Start the job
217
client.fine_tuning.start(job.id)
218

219
# Monitor progress
220
while True:
221
    job_status = client.fine_tuning.get(job.id)
222
    print(f"Job status: {job_status.status}")
223
    
224
    if job_status.status in ["SUCCEEDED", "FAILED", "CANCELLED"]:
225
        break
226
    
227
    time.sleep(30)  # Check every 30 seconds
228

229
if job_status.status == "SUCCEEDED":
230
    print(f"Fine-tuned model ID: {job_status.fine_tuned_model}")
231
    print("Fine-tuning completed successfully!")
232
else:
233
    print(f"Fine-tuning failed with status: {job_status.status}")
234
```
235

236
### Use Fine-tuned Model
237

238
```python
239
from mistralai.models import UserMessage
240

241
# Use the fine-tuned model
242
if job_status.status == "SUCCEEDED":
243
    response = client.chat.complete(
244
        model=job_status.fine_tuned_model,
245
        messages=[UserMessage(content="What is machine learning?")],
246
        temperature=0.7
247
    )
248
    
249
    print("Fine-tuned model response:")
250
    print(response.choices[0].message.content)
251
```
252

253
### Integration with Weights & Biases
254

255
```python
256
from mistralai.models import WandbIntegration
257

258
# Create job with W&B integration
259
wandb_integration = WandbIntegration(
260
    type="wandb",
261
    project="my-fine-tuning-project",
262
    name="experiment-1",
263
    api_key="your-wandb-api-key"
264
)
265

266
job = client.fine_tuning.create(
267
    model="mistral-small-latest",
268
    training_files=[training_file],
269
    integrations=[wandb_integration],
270
    suffix="wandb-tracked-model"
271
)
272
```
273

274
## Types
275

276
### Job Types
277

278
```python { .api }
279
class CompletionDetailedJobOut:
280
    id: str
281
    object: str
282
    model: str
283
    status: str
284
    job_type: str
285
    created_at: int
286
    modified_at: int
287
    training_files: List[TrainingFile]
288
    validation_files: Optional[List[TrainingFile]]
289
    hyperparameters: dict
290
    fine_tuned_model: Optional[str]
291
    integrations: Optional[List[dict]]
292
    trained_tokens: Optional[int]
293
    epochs: Optional[int]
294
    expected_duration_seconds: Optional[int]
295

296
class JobsOut:
297
    data: List[CompletionJobOut]
298
    object: str
299
    total: int
300

301
class CompletionJobOut:
302
    id: str
303
    object: str
304
    model: str
305
    status: str
306
    job_type: str
307
    created_at: int
308
    modified_at: int
309
    fine_tuned_model: Optional[str]
310
```
311

312
### Training Configuration
313

314
```python { .api }
315
class TrainingFile:
316
    file_id: str
317
    weight: Optional[float]
318

319
class CompletionTrainingParameters:
320
    training_steps: Optional[int]
321
    learning_rate: Optional[float]
322
    weight_decay: Optional[float]
323
    warmup_fraction: Optional[float]
324
    epochs: Optional[int]
325
    fim_ratio: Optional[float]
326
    seq_len: Optional[int]
327
```
328

329
### Model Types
330

331
```python { .api }
332
class CompletionFTModelOut:
333
    id: str
334
    object: str
335
    created: int
336
    owned_by: str
337
    root: str
338
    archived: bool
339
    name: Optional[str]
340
    description: Optional[str]
341
    max_context_length: Optional[int]
342
    aliases: Optional[List[str]]
343
    capabilities: Optional[List[str]]
344
    job: str
345

346
class ArchiveFTModelOut:
347
    id: str
348
    object: str
349
    archived: bool
350

351
class UnarchiveFTModelOut:
352
    id: str
353
    object: str
354
    archived: bool
355
```
356

357
### Job Status Values
358

359
- **QUEUED**: Job is waiting to start
360
- **RUNNING**: Job is currently training
361
- **SUCCEEDED**: Job completed successfully
362
- **FAILED**: Job failed during training
363
- **CANCELLED**: Job was cancelled by user
364
- **CANCELLING**: Job is in the process of being cancelled
365

366
### Best Practices
367

368
- Use high-quality, diverse training data
369
- Start with smaller datasets to test hyperparameters
370
- Monitor training metrics through integrations
371
- Validate model performance before production use
372
- Archive unused models to manage costs
373
- Use descriptive suffixes for model identification