0
# Marshal System
1

2
The Marshal system provides advanced type conversion capabilities between protocol buffers and Python native types. It handles automatic marshaling of well-known types, custom type conversions, and provides extensibility for application-specific type transformations.
3

4
## Capabilities
5

6
### Marshal Creation and Management
7

8
Create and manage marshal instances for type conversion between protocol buffers and Python objects.
9

10
```python { .api }
11
class Marshal:
12
    """
13
    The translator between protocol buffer and Python instances.
14
    
15
    Multiple instantiations with the same name provide the same instance,
16
    enabling shared marshal registries across modules.
17
    
18
    Args:
19
        name (str): The name of the marshal. Multiple marshals with the same
20
                   name argument will provide the same marshal instance.
21
    """
22
    def __init__(self, *, name: str): ...
23
    
24
    def register(self, proto_type: type, rule: Rule = None): ...
25
    def to_python(self, proto_type, value, *, absent: bool = None): ...
26
    def to_proto(self, proto_type, value, *, strict: bool = False): ...
27
    def reset(self): ...
28
```
29

30
Example usage:
31

32
```python
33
# Create a marshal instance
34
marshal = proto.Marshal(name="my_app")
35

36
# Multiple references to the same marshal
37
marshal2 = proto.Marshal(name="my_app")  # Same instance as marshal
38

39
# Each proto-plus message automatically gets its own marshal
40
# based on the module/package structure
41
```
42

43
### Type Registration
44

45
Register custom conversion rules for specific protocol buffer types.
46

47
```python { .api }
48
def register(self, proto_type: type, rule: Rule = None):
49
    """
50
    Register a rule against the given proto_type.
51
    
52
    Args:
53
        proto_type (type): A protocol buffer message type.
54
        rule: A marshal object with to_python and to_proto methods.
55
        
56
    Can also be used as a decorator for rule classes.
57
    """
58
```
59

60
Example usage:
61

62
```python
63
from google.protobuf import timestamp_pb2
64
import datetime
65

66
# Create a custom rule class
67
class CustomTimestampRule:
68
    def to_python(self, pb_value, *, absent: bool = None):
69
        if absent:
70
            return None
71
        return datetime.datetime.fromtimestamp(
72
            pb_value.seconds + pb_value.nanos / 1e9
73
        )
74
    
75
    def to_proto(self, py_value):
76
        if py_value is None:
77
            return timestamp_pb2.Timestamp()
78
        timestamp = int(py_value.timestamp())
79
        return timestamp_pb2.Timestamp(seconds=timestamp)
80

81
# Register the rule
82
marshal = proto.Marshal(name="custom")
83
marshal.register(timestamp_pb2.Timestamp, CustomTimestampRule())
84

85
# Or use as a decorator
86
@marshal.register(timestamp_pb2.Timestamp)
87
class TimestampRule:
88
    def to_python(self, pb_value, *, absent: bool = None):
89
        # Custom conversion logic
90
        pass
91
    
92
    def to_proto(self, py_value):
93
        # Custom conversion logic  
94
        pass
95
```
96

97
### Python Type Conversion
98

99
Convert protocol buffer values to appropriate Python types.
100

101
```python { .api }
102
def to_python(self, proto_type, value, *, absent: bool = None):
103
    """
104
    Convert a protocol buffer value to the appropriate Python type.
105
    
106
    Args:
107
        proto_type: The protocol buffer type descriptor
108
        value: The protocol buffer value to convert
109
        absent (bool): Whether the field was absent in the message
110
        
111
    Returns:
112
        The converted Python value
113
    """
114
```
115

116
Example usage:
117

118
```python
119
from google.protobuf import timestamp_pb2
120
import datetime
121

122
marshal = proto.Marshal(name="example")
123

124
# Convert timestamp to Python datetime
125
pb_timestamp = timestamp_pb2.Timestamp()
126
pb_timestamp.GetCurrentTime()
127

128
py_datetime = marshal.to_python(timestamp_pb2.Timestamp, pb_timestamp)
129
print(type(py_datetime))  # <class 'datetime.datetime'>
130

131
# Convert repeated fields to Python lists
132
# (handled automatically by the marshal)
133
```
134

135
### Protocol Buffer Conversion
136

137
Convert Python values to protocol buffer format.
138

139
```python { .api }
140
def to_proto(self, proto_type, value, *, strict: bool = False):
141
    """
142
    Convert a Python value to the appropriate protocol buffer type.
143
    
144
    Args:
145
        proto_type: The target protocol buffer type
146
        value: The Python value to convert
147
        strict (bool): If True, require exact type match
148
        
149
    Returns:
150
        The converted protocol buffer value
151
    """
152
```
153

154
Example usage:
155

156
```python
157
from google.protobuf import timestamp_pb2
158
import datetime
159

160
marshal = proto.Marshal(name="example")
161

162
# Convert Python datetime to timestamp
163
py_datetime = datetime.datetime.now()
164
pb_timestamp = marshal.to_proto(timestamp_pb2.Timestamp, py_datetime)
165
print(type(pb_timestamp))  # <class 'google.protobuf.timestamp_pb2.Timestamp'>
166

167
# Strict mode validation
168
try:
169
    result = marshal.to_proto(timestamp_pb2.Timestamp, "invalid", strict=True)
170
except TypeError as e:
171
    print(f"Type mismatch: {e}")
172
```
173

174
### Marshal Reset
175

176
Reset the marshal to its default state with built-in rules.
177

178
```python { .api }
179
def reset(self):
180
    """Reset the registry to its initial state."""
181
```
182

183
Example usage:
184

185
```python
186
marshal = proto.Marshal(name="example")
187

188
# Add custom rules
189
marshal.register(SomeType, CustomRule())
190

191
# Reset to default rules only
192
marshal.reset()
193
```
194

195
## Built-in Type Conversions
196

197
### Well-Known Types
198

199
Proto-plus automatically handles conversion for common protocol buffer well-known types:
200

201
```python
202
from google.protobuf import timestamp_pb2, duration_pb2, wrappers_pb2
203
import datetime
204

205
class Event(proto.Message):
206
    # Timestamp converts to/from datetime.datetime
207
    created_at = proto.Field(timestamp_pb2.Timestamp, number=1)
208
    
209
    # Duration converts to/from datetime.timedelta  
210
    duration = proto.Field(duration_pb2.Duration, number=2)
211
    
212
    # Wrapper types convert to/from native Python types
213
    description = proto.Field(wrappers_pb2.StringValue, number=3)
214
    count = proto.Field(wrappers_pb2.Int32Value, number=4)
215

216
# Usage with automatic conversion
217
event = Event(
218
    created_at=datetime.datetime.now(),           # datetime -> Timestamp
219
    duration=datetime.timedelta(hours=2),         # timedelta -> Duration  
220
    description="Sample event",                   # str -> StringValue
221
    count=42                                      # int -> Int32Value
222
)
223

224
# Access converted values
225
print(type(event.created_at))    # <class 'datetime.datetime'>
226
print(type(event.duration))      # <class 'datetime.timedelta'>
227
print(type(event.description))   # <class 'str'>
228
print(type(event.count))         # <class 'int'>
229
```
230

231
### Struct and Value Types
232

233
Automatic conversion for google.protobuf.Struct and google.protobuf.Value:
234

235
```python
236
from google.protobuf import struct_pb2
237

238
class ApiResponse(proto.Message):
239
    # Struct converts to/from Python dict
240
    metadata = proto.Field(struct_pb2.Struct, number=1)
241
    
242
    # Value converts to/from Python native types
243
    result = proto.Field(struct_pb2.Value, number=2)
244

245
# Usage with automatic conversion
246
response = ApiResponse(
247
    metadata={
248
        "request_id": "12345",
249
        "timestamp": 1640995200,
250
        "success": True
251
    },
252
    result=[1, 2, 3, "test"]  # Converted to appropriate Value type
253
)
254

255
# Access as native Python types
256
print(response.metadata["success"])  # True (bool)
257
print(response.result[3])            # "test" (str)
258
```
259

260
### Field Mask Types
261

262
Automatic conversion for google.protobuf.FieldMask:
263

264
```python
265
from google.protobuf import field_mask_pb2
266

267
class UpdateRequest(proto.Message):
268
    # FieldMask converts to/from list of field paths
269
    update_mask = proto.Field(field_mask_pb2.FieldMask, number=1)
270

271
# Usage
272
request = UpdateRequest(
273
    update_mask=["name", "email", "address.city"]  # List -> FieldMask
274
)
275

276
# Access as list
277
for path in request.update_mask:
278
    print(f"Update path: {path}")
279
```
280

281
## Advanced Marshal Features
282

283
### Collection Handling
284

285
The marshal system automatically handles protocol buffer collections:
286

287
```python
288
class DataSet(proto.Message):
289
    # Repeated fields become Python lists
290
    values = proto.RepeatedField(proto.FLOAT, number=1)
291
    
292
    # Map fields become Python dictionaries  
293
    labels = proto.MapField(proto.STRING, proto.STRING, number=2)
294

295
data = DataSet(
296
    values=[1.0, 2.5, 3.7],
297
    labels={"category": "test", "version": "1.0"}
298
)
299

300
# Collections behave like native Python types
301
data.values.append(4.2)
302
data.labels["new_key"] = "new_value"
303

304
print(len(data.values))      # 4
305
print(list(data.labels.keys()))  # ["category", "version", "new_key"]
306
```
307

308
### Bytes and String Handling
309

310
Special handling for bytes fields with base64 encoding support:
311

312
```python
313
class FileData(proto.Message):
314
    filename = proto.Field(proto.STRING, number=1)
315
    content = proto.Field(proto.BYTES, number=2)
316

317
# Bytes fields accept bytes or base64 strings
318
file_data = FileData(
319
    filename="example.txt",
320
    content=b"Hello, World!"  # Raw bytes
321
)
322

323
# Can also use base64 strings in JSON
324
json_data = '{"filename": "test.txt", "content": "SGVsbG8sIFdvcmxkIQ=="}'
325
file_from_json = FileData.from_json(json_data)
326
print(file_from_json.content)  # b'Hello, World!'
327
```
328

329
### Custom Rule Implementation
330

331
Implement custom conversion rules for application-specific types:
332

333
```python
334
from proto.marshal import Rule
335

336
class ColorRule(Rule):
337
    """Convert between RGB tuples and color messages."""
338
    
339
    def to_python(self, pb_value, *, absent: bool = None):
340
        if absent:
341
            return None
342
        return (pb_value.red, pb_value.green, pb_value.blue)
343
    
344
    def to_proto(self, py_value):
345
        if isinstance(py_value, tuple) and len(py_value) == 3:
346
            return ColorMessage(red=py_value[0], green=py_value[1], blue=py_value[2])
347
        return py_value
348

349
# Register and use the custom rule
350
marshal = proto.Marshal(name="graphics")
351
marshal.register(ColorMessage, ColorRule())
352
```