0
# Replicate
1

2
A comprehensive Python client library for the Replicate platform, enabling developers to run AI models, manage predictions, handle file outputs, and interact with the Replicate API programmatically. It offers both synchronous and asynchronous APIs for running models locally or in the cloud, supports streaming outputs for real-time model results, provides file handling capabilities with efficient FileOutput objects for managing model-generated files, and includes comprehensive prediction management with background execution, webhooks, and cancellation support.
3

4
## Package Information
5

6
- **Package Name**: replicate
7
- **Language**: Python
8
- **Installation**: `pip install replicate`
9
- **Minimum Python Version**: 3.8+
10

11
## Core Imports
12

13
```python
14
import replicate
15
```
16

17
For client-based usage:
18

19
```python
20
from replicate import Client
21
from replicate.exceptions import ModelError, ReplicateError
22
```
23

24
## Basic Usage
25

26
### Quick Start - Run a Model
27

28
```python
29
import replicate
30

31
# Set your API token as environment variable: REPLICATE_API_TOKEN=<your token>
32

33
# Run a model and get output
34
outputs = replicate.run(
35
    "black-forest-labs/flux-schnell",
36
    input={"prompt": "astronaut riding a rocket like a horse"}
37
)
38

39
# Handle file outputs
40
for index, output in enumerate(outputs):
41
    with open(f"output_{index}.webp", "wb") as file:
42
        file.write(output.read())
43
```
44

45
### Stream Model Output
46

47
```python
48
import replicate
49

50
# Stream tokens as they are generated
51
for event in replicate.stream(
52
    "meta/meta-llama-3-70b-instruct",
53
    input={"prompt": "Please write a haiku about llamas."}
54
):
55
    print(str(event), end="")
56
```
57

58
### Background Predictions
59

60
```python
61
import replicate
62

63
# Start a prediction in the background
64
model = replicate.models.get("kvfrans/clipdraw")
65
version = model.versions.get("5797a99edc939ea0e9242d5e8c9cb3bc7d125b1eac21bda852e5cb79ede2cd9b")
66
prediction = replicate.predictions.create(
67
    version=version,
68
    input={"prompt": "Watercolor painting of an underwater submarine"}
69
)
70

71
# Check status and wait for completion
72
print(prediction.status)  # 'starting'
73
prediction.wait()
74
print(prediction.status)  # 'succeeded'
75

76
# Access output
77
with open("output.png", "wb") as file:
78
    file.write(prediction.output.read())
79
```
80

81
## Architecture
82

83
The Replicate client is built around several key components:
84

85
- **Client**: Main interface for API interaction with configurable authentication and HTTP settings
86
- **Resources**: Model, Prediction, Training, Collection, Deployment, File objects representing API entities
87
- **Namespaces**: Organized API endpoints (models, predictions, trainings, etc.) accessed via client properties
88
- **FileOutput**: Efficient file handling for model outputs with streaming and URL access
89
- **Streaming**: Server-sent events for real-time model output consumption
90
- **Pagination**: Automated handling of paginated API responses
91

92
## Capabilities
93

94
### Core Model Execution
95

96
Primary functions for running AI models synchronously and asynchronously, with support for streaming outputs and file handling.
97

98
```python { .api }
99
def run(
100
    ref: str,
101
    input: Optional[Dict[str, Any]] = None,
102
    *,
103
    use_file_output: Optional[bool] = True,
104
    **params
105
) -> Union[Any, Iterator[Any]]: ...
106

107
async def async_run(
108
    ref: str,
109
    input: Optional[Dict[str, Any]] = None,
110
    *,
111
    use_file_output: Optional[bool] = True,
112
    **params
113
) -> Union[Any, AsyncIterator[Any]]: ...
114

115
def stream(
116
    ref: str,
117
    *,
118
    input: Optional[Dict[str, Any]] = None,
119
    use_file_output: Optional[bool] = True,
120
    **params
121
) -> Iterator[ServerSentEvent]: ...
122

123
async def async_stream(
124
    ref: str,
125
    input: Optional[Dict[str, Any]] = None,
126
    *,
127
    use_file_output: Optional[bool] = True,
128
    **params
129
) -> AsyncIterator[ServerSentEvent]: ...
130
```
131

132
[Client & Authentication](./client.md)
133

134
### Model and Prediction Management
135

136
Comprehensive management of models, versions, and prediction lifecycle including creation, monitoring, cancellation, and output retrieval.
137

138
```python { .api }
139
# Namespace access
140
accounts: Accounts
141
models: Models
142
predictions: Predictions
143

144
# Core prediction operations
145
def predictions.create(
146
    model: Optional[str] = None,
147
    version: Optional[str] = None,
148
    input: Optional[Dict[str, Any]] = None,
149
    **params
150
) -> Prediction: ...
151

152
def predictions.get(id: str) -> Prediction: ...
153
def predictions.list(**params) -> Page[Prediction]: ...
154
def predictions.cancel(id: str) -> Prediction: ...
155
```
156

157
[Models & Predictions](./models-predictions.md)
158

159
### File Handling and Outputs
160

161
Efficient handling of file inputs and outputs with FileOutput objects that provide streaming, URL access, and metadata.
162

163
```python { .api }
164
class FileOutput:
165
    def read() -> bytes: ...
166
    def __iter__() -> Iterator[bytes]: ...
167
    async def aread() -> bytes: ...
168
    async def __aiter__() -> AsyncIterator[bytes]: ...
169
    @property
170
    def url() -> str: ...
171

172
# File management namespace
173
files: Files
174

175
def files.create(file: BinaryIO, **params) -> File: ...
176
def files.get(id: str) -> File: ...
177
def files.list(**params) -> Page[File]: ...
178
```
179

180
[File Handling](./files.md)
181

182
### Collections and Training
183

184
Management of model collections and training operations for custom model development.
185

186
```python { .api }
187
# Collection operations
188
collections: Collections
189
def collections.get(name: str) -> Collection: ...
190
def collections.list(**params) -> Page[Collection]: ...
191

192
# Training operations  
193
trainings: Trainings
194
def trainings.create(
195
    model: str,
196
    version: str,
197
    input: Dict[str, Any],
198
    **params
199
) -> Training: ...
200
def trainings.get(id: str) -> Training: ...
201
def trainings.list(**params) -> Page[Training]: ...
202
```
203

204
[Collections & Training](./collections-training.md)
205

206
### Deployments and Webhooks
207

208
Management of model deployments and webhook configuration for production use cases.
209

210
```python { .api }
211
# Deployment operations
212
deployments: Deployments
213
def deployments.get(owner: str, name: str) -> Deployment: ...
214
def deployments.list(**params) -> Page[Deployment]: ...
215

216
# Webhook operations
217
webhooks: Webhooks
218
def webhooks.default.secret() -> WebhookSigningSecret: ...
219
@staticmethod
220
def webhooks.validate(
221
    request: Optional[httpx.Request] = None,
222
    headers: Optional[Dict[str, str]] = None,
223
    body: Optional[str] = None,
224
    secret: Optional[WebhookSigningSecret] = None,
225
    tolerance: Optional[int] = None
226
) -> None: ...
227
```
228

229
[Deployments & Webhooks](./deployments-webhooks.md)
230

231
### Pagination Utilities
232

233
Helper functions for handling paginated API responses.
234

235
```python { .api }
236
def paginate(pages, **kwargs): ...
237
async def async_paginate(pages, **kwargs): ...
238

239
class Page[T]:
240
    results: List[T]
241
    next: Optional[str]
242
    previous: Optional[str]
243
```
244

245
### Error Handling
246

247
Comprehensive exception classes for handling API errors and model failures.
248

249
```python { .api }
250
class ReplicateException(Exception): ...
251
class ModelError(ReplicateException):
252
    prediction: Prediction
253
class ReplicateError(ReplicateException):
254
    type: Optional[str]
255
    title: Optional[str]
256
    status: Optional[int]
257
    detail: Optional[str]
258
    instance: Optional[str]
259
```
260

261
[Error Handling](./exceptions.md)
262

263
## Types
264

265
```python { .api }
266
from typing import Any, Dict, List, Optional, Union, Iterator, AsyncIterator
267
from typing_extensions import Literal
268

269
# Server-sent event for streaming
270
class ServerSentEvent:
271
    event: Optional[str]
272
    data: Optional[str]
273
    id: Optional[str]
274
    retry: Optional[int]
275

276
# File encoding strategies
277
FileEncodingStrategy = Literal["base64", "url"]
278

279
# Prediction status types
280
PredictionStatus = Literal["starting", "processing", "succeeded", "failed", "canceled"]
281

282
# Model visibility types  
283
ModelVisibility = Literal["public", "private"]
284
```