pypi-pydantic-ai

1
# Messages and Media
2

3
Rich message system supporting text, images, audio, video, documents, and binary content. Includes comprehensive streaming support and delta updates for real-time interactions.
4

5
## Capabilities
6

7
### Message Structure
8

9
Core message types for agent-model communication.
10

11
```python { .api }
12
class ModelRequest:
13
    """
14
    Request message sent to a model.
15
    """
16
    parts: list[ModelRequestPart]
17
    kind: Literal['request']
18

19
class ModelResponse:
20
    """
21
    Response message received from a model.
22
    """
23
    parts: list[ModelResponsePart]
24
    timestamp: datetime
25
    kind: Literal['response']
26

27
ModelMessage = ModelRequest | ModelResponse
28

29
class ModelMessagesTypeAdapter:
30
    """
31
    Type adapter for serializing/deserializing model messages.
32
    """
33
    def validate_python(self, data: Any) -> list[ModelMessage]: ...
34
    def dump_python(self, messages: list[ModelMessage]) -> list[dict]: ...
35
```
36

37
### Request Message Parts
38

39
Parts that can be included in requests to models.
40

41
```python { .api }
42
class SystemPromptPart:
43
    """System prompt message part."""
44
    content: str
45
    kind: Literal['system-prompt']
46

47
class UserPromptPart:
48
    """User prompt message part."""
49
    content: str | list[UserContent]
50
    timestamp: datetime
51
    kind: Literal['user-prompt']
52

53
class ToolReturnPart:
54
    """Tool return message part."""
55
    tool_name: str
56
    content: Any
57
    tool_id: str | None
58
    timestamp: datetime
59
    kind: Literal['tool-return']
60

61
class BuiltinToolReturnPart:
62
    """Built-in tool return message part."""
63
    tool_name: str
64
    content: Any
65
    tool_id: str | None
66
    timestamp: datetime
67
    kind: Literal['builtin-tool-return']
68

69
class RetryPromptPart:
70
    """Retry prompt message part."""
71
    content: str
72
    tool_name: str | None
73
    tool_id: str | None
74
    timestamp: datetime
75
    kind: Literal['retry-prompt']
76

77
ModelRequestPart = (
78
    SystemPromptPart |
79
    UserPromptPart |
80
    ToolReturnPart |
81
    BuiltinToolReturnPart |
82
    RetryPromptPart
83
)
84
```
85

86
### Response Message Parts
87

88
Parts that can be included in responses from models.
89

90
```python { .api }
91
class TextPart:
92
    """Plain text response part."""
93
    content: str
94
    kind: Literal['text']
95

96
class ThinkingPart:
97
    """Thinking response part for reasoning models like o1."""
98
    content: str
99
    kind: Literal['thinking']
100

101
class ToolCallPart:
102
    """Tool call request part."""
103
    tool_name: str
104
    args: dict[str, Any]
105
    tool_id: str | None
106
    kind: Literal['tool-call']
107

108
class BuiltinToolCallPart:
109
    """Built-in tool call part."""
110
    tool_name: str
111
    args: dict[str, Any]
112
    tool_id: str | None
113
    kind: Literal['builtin-tool-call']
114

115
ModelResponsePart = (
116
    TextPart |
117
    ThinkingPart |
118
    ToolCallPart |
119
    BuiltinToolCallPart
120
)
121
```
122

123
### Media and File Types
124

125
Support for various media types and file content.
126

127
```python { .api }
128
class FileUrl:
129
    """Abstract base for URL-based files."""
130
    url: str
131

132
class ImageUrl(FileUrl):
133
    """Image file URL with format and media type."""
134
    def __init__(
135
        self,
136
        url: str,
137
        *,
138
        alt: str | None = None,
139
        media_type: ImageMediaType | None = None
140
    ):
141
        """
142
        Create image URL.
143
        
144
        Parameters:
145
        - url: URL to image file
146
        - alt: Alt text for accessibility
147
        - media_type: MIME type of image
148
        """
149

150
class AudioUrl(FileUrl):
151
    """Audio file URL with format and media type."""
152
    def __init__(
153
        self,
154
        url: str,
155
        *,
156
        media_type: AudioMediaType | None = None
157
    ):
158
        """
159
        Create audio URL.
160
        
161
        Parameters:
162
        - url: URL to audio file
163
        - media_type: MIME type of audio
164
        """
165

166
class VideoUrl(FileUrl):
167
    """Video file URL with format and media type."""
168
    def __init__(
169
        self,
170
        url: str,
171
        *,
172
        media_type: VideoMediaType | None = None
173
    ):
174
        """
175
        Create video URL.
176
        
177
        Parameters:
178
        - url: URL to video file
179
        - media_type: MIME type of video
180
        """
181

182
class DocumentUrl(FileUrl):
183
    """Document file URL with format."""
184
    def __init__(
185
        self,
186
        url: str,
187
        *,
188
        media_type: DocumentMediaType | None = None
189
    ):
190
        """
191
        Create document URL.
192
        
193
        Parameters:
194
        - url: URL to document file
195
        - media_type: MIME type of document
196
        """
197

198
class BinaryContent:
199
    """Binary file content with format and media type."""
200
    def __init__(
201
        self,
202
        data: bytes,
203
        media_type: str,
204
        *,
205
        filename: str | None = None
206
    ):
207
        """
208
        Create binary content.
209
        
210
        Parameters:
211
        - data: Binary file data
212
        - media_type: MIME type of content
213
        - filename: Optional filename
214
        """
215
```
216

217
### Media Type Definitions
218

219
Type definitions for supported media formats.
220

221
```python { .api }
222
AudioMediaType = Literal[
223
    'audio/mpeg',
224
    'audio/wav',
225
    'audio/ogg',
226
    'audio/mp4',
227
    'audio/webm',
228
    'audio/flac'
229
]
230

231
ImageMediaType = Literal[
232
    'image/jpeg',
233
    'image/png',
234
    'image/gif',
235
    'image/webp',
236
    'image/svg+xml',
237
    'image/bmp'
238
]
239

240
VideoMediaType = Literal[
241
    'video/mp4',
242
    'video/webm',
243
    'video/ogg',
244
    'video/avi',
245
    'video/mov',
246
    'video/wmv'
247
]
248

249
DocumentMediaType = Literal[
250
    'application/pdf',
251
    'text/plain',
252
    'text/html',
253
    'text/markdown',
254
    'application/msword',
255
    'application/vnd.openxmlformats-officedocument.wordprocessingml.document'
256
]
257

258
AudioFormat = Literal['mp3', 'wav', 'ogg', 'mp4', 'webm', 'flac']
259
ImageFormat = Literal['jpeg', 'png', 'gif', 'webp', 'svg', 'bmp']
260
VideoFormat = Literal['mp4', 'webm', 'ogg', 'avi', 'mov', 'wmv']
261
DocumentFormat = Literal['pdf', 'txt', 'html', 'md', 'doc', 'docx']
262
```
263

264
### User Content Types
265

266
Types of content that can be included in user messages.
267

268
```python { .api }
269
UserContent = (
270
    str |
271
    ImageUrl |
272
    AudioUrl |
273
    VideoUrl |
274
    DocumentUrl |
275
    BinaryContent
276
)
277
```
278

279
### Tool Return Handling
280

281
Structured tool return objects for complex tool outputs.
282

283
```python { .api }
284
class ToolReturn:
285
    """
286
    Structured tool return with content.
287
    """
288
    def __init__(
289
        self,
290
        content: Any,
291
        *,
292
        metadata: dict[str, Any] | None = None
293
    ):
294
        """
295
        Create tool return.
296
        
297
        Parameters:
298
        - content: Return content from tool
299
        - metadata: Optional metadata about the return
300
        """
301
```
302

303
### Streaming Events
304

305
Event types for streaming responses and real-time updates.
306

307
```python { .api }
308
class PartStartEvent:
309
    """New part started event."""
310
    part: ModelResponsePart
311
    kind: Literal['part-start']
312

313
class PartDeltaEvent:
314
    """Part delta update event."""
315
    delta: ModelResponsePartDelta
316
    kind: Literal['part-delta']
317

318
class FinalResultEvent:
319
    """Final result ready event."""
320
    result: Any
321
    kind: Literal['final-result']
322

323
class FunctionToolCallEvent:
324
    """Function tool call event."""
325
    tool_name: str
326
    args: dict[str, Any]
327
    tool_id: str | None
328
    kind: Literal['function-tool-call']
329

330
class FunctionToolResultEvent:
331
    """Function tool result event."""
332
    tool_name: str
333
    result: Any
334
    tool_id: str | None
335
    kind: Literal['function-tool-result']
336

337
class BuiltinToolCallEvent:
338
    """Built-in tool call event."""
339
    tool_name: str
340
    args: dict[str, Any]
341
    tool_id: str | None
342
    kind: Literal['builtin-tool-call']
343

344
class BuiltinToolResultEvent:
345
    """Built-in tool result event."""
346
    tool_name: str
347
    result: Any
348
    tool_id: str | None
349
    kind: Literal['builtin-tool-result']
350

351
ModelResponseStreamEvent = (
352
    PartStartEvent |
353
    PartDeltaEvent
354
)
355

356
HandleResponseEvent = (
357
    FunctionToolCallEvent |
358
    FunctionToolResultEvent |
359
    BuiltinToolCallEvent |
360
    BuiltinToolResultEvent
361
)
362

363
AgentStreamEvent = (
364
    ModelResponseStreamEvent |
365
    HandleResponseEvent |
366
    FinalResultEvent
367
)
368
```
369

370
### Delta Types for Streaming
371

372
Delta update types for streaming responses.
373

374
```python { .api }
375
class TextPartDelta:
376
    """Text part delta update."""
377
    content: str
378
    kind: Literal['text']
379

380
class ThinkingPartDelta:
381
    """Thinking part delta update."""
382
    content: str
383
    kind: Literal['thinking']
384

385
class ToolCallPartDelta:
386
    """Tool call part delta update."""
387
    tool_name: str | None
388
    args: dict[str, Any] | None
389
    tool_id: str | None
390
    kind: Literal['tool-call']
391

392
ModelResponsePartDelta = (
393
    TextPartDelta |
394
    ThinkingPartDelta |
395
    ToolCallPartDelta
396
)
397
```
398

399
## Usage Examples
400

401
### Basic Text Messages
402

403
```python
404
from pydantic_ai import Agent
405

406
agent = Agent(model='gpt-4', system_prompt='You are helpful.')
407

408
# Simple text interaction
409
result = agent.run_sync('Hello, how are you?')
410
print(result.data)
411
```
412

413
### Images in Messages
414

415
```python
416
from pydantic_ai import Agent, ImageUrl
417

418
agent = Agent(
419
    model='gpt-4-vision-preview',
420
    system_prompt='You can analyze images.'
421
)
422

423
# Send image with message
424
image = ImageUrl('https://example.com/image.jpg', alt='A sample image')
425
result = agent.run_sync(['Describe this image:', image])
426
print(result.data)
427
```
428

429
### Multiple Media Types
430

431
```python
432
from pydantic_ai import Agent, ImageUrl, AudioUrl, DocumentUrl
433

434
agent = Agent(
435
    model='gpt-4-turbo',
436
    system_prompt='You can process multiple media types.'
437
)
438

439
# Mix of text and media
440
content = [
441
    'Please analyze these files:',
442
    ImageUrl('https://example.com/chart.png'),
443
    AudioUrl('https://example.com/recording.mp3'),
444
    DocumentUrl('https://example.com/report.pdf')
445
]
446

447
result = agent.run_sync(content)
448
```
449

450
### Binary Content
451

452
```python
453
from pydantic_ai import Agent, BinaryContent
454

455
agent = Agent(model='gpt-4-vision-preview')
456

457
# Load image file as binary
458
with open('image.jpg', 'rb') as f:
459
    image_data = f.read()
460

461
binary_image = BinaryContent(
462
    data=image_data,
463
    media_type='image/jpeg',
464
    filename='image.jpg'
465
)
466

467
result = agent.run_sync(['Analyze this image:', binary_image])
468
```
469

470
### Message History
471

472
```python
473
from pydantic_ai import Agent
474
from pydantic_ai.messages import ModelRequest, UserPromptPart
475

476
agent = Agent(model='gpt-4')
477

478
# Create message history
479
message_history = [
480
    ModelRequest([
481
        UserPromptPart('What is the capital of France?')
482
    ])
483
]
484

485
# Continue conversation with history
486
result = agent.run_sync(
487
    'What about Germany?',
488
    message_history=message_history
489
)
490
```
491

492
### Streaming with Events
493

494
```python
495
from pydantic_ai import Agent
496

497
agent = Agent(model='gpt-4')
498

499
async def stream_example():
500
    stream = await agent.run_stream('Tell me a story')
501
    
502
    async for event in stream:
503
        if event.kind == 'part-delta':
504
            print(event.delta.content, end='', flush=True)
505
        elif event.kind == 'final-result':
506
            print(f"\n\nFinal result: {event.result}")
507
            break
508

509
# Run streaming example
510
import asyncio
511
asyncio.run(stream_example())
512
```