0
# msgpack-numpy
1

2
A library that enables serialization and deserialization of numpy arrays and scalar data types using the highly efficient msgpack format. The package provides both automatic monkey-patching functionality and manual encoder/decoder functions, ensuring preservation of numerical data types and precision during serialization/deserialization operations.
3

4
## Package Information
5

6
- **Package Name**: msgpack-numpy
7
- **Package Type**: pypi
8
- **Language**: Python
9
- **Installation**: `pip install msgpack-numpy`
10

11
## Core Imports
12

13
```python
14
import msgpack_numpy as m
15
```
16

17
For manual encoding/decoding:
18

19
```python
20
import msgpack_numpy as m
21
import msgpack
22
```
23

24
## Basic Usage
25

26
### Monkey Patching Approach (Automatic)
27

28
The easiest way to use msgpack-numpy is to call its monkey patching function:
29

30
```python
31
import msgpack
32
import msgpack_numpy as m
33
import numpy as np
34

35
# Enable automatic numpy support
36
m.patch()
37

38
# Now all msgpack operations are numpy-aware
39
x = np.random.rand(5)
40
x_packed = msgpack.packb(x)
41
x_unpacked = msgpack.unpackb(x_packed)
42
```
43

44
### Manual Encoding/Decoding
45

46
For explicit control over the serialization process:
47

48
```python
49
import msgpack
50
import msgpack_numpy as m
51
import numpy as np
52

53
x = np.random.rand(5)
54
x_packed = msgpack.packb(x, default=m.encode)
55
x_unpacked = msgpack.unpackb(x_packed, object_hook=m.decode)
56
```
57

58
### Using Direct Pack/Unpack Functions
59

60
```python
61
import msgpack_numpy as m
62
import numpy as np
63

64
x = np.array([1, 2, 3], dtype=np.int32)
65
x_packed = m.packb(x)
66
x_unpacked = m.unpackb(x_packed)
67
```
68

69
## Capabilities
70

71
### Core Encoder/Decoder Functions
72

73
Functions for explicit serialization and deserialization of numpy data types.
74

75
```python { .api }
76
def encode(obj, chain=None):
77
    """
78
    Data encoder for serializing numpy data types.
79
    
80
    Parameters:
81
    - obj: Object to encode (numpy arrays, scalars, complex numbers, or other objects)
82
    - chain: Optional chained encoder function for non-numpy objects
83
    
84
    Returns:
85
    Dictionary with serialization metadata or original object if not numpy type
86
    """
87

88
def decode(obj, chain=None):
89
    """
90
    Decoder for deserializing numpy data types.
91
    
92
    Parameters:
93
    - obj: Object to decode (serialized dictionary or other objects)
94
    - chain: Optional chained decoder function for non-numpy objects
95
    
96
    Returns:
97
    Reconstructed numpy array/scalar or original object if not serialized numpy type
98
    """
99
```
100

101
### Packer and Unpacker Classes
102

103
Msgpack classes with automatic numpy support built-in.
104

105
```python { .api }
106
class Packer:
107
    """
108
    Msgpack packer with automatic numpy encoding support.
109
    Inherits from msgpack.Packer with numpy-aware default encoder.
110
    """
111
    def __init__(self, default=None, **kwargs):
112
        """
113
        Initialize packer with numpy support.
114
        
115
        Parameters:
116
        - default: Optional additional default encoder (chained with numpy encoder)
117
        - **kwargs: Additional msgpack.Packer parameters (version-dependent)
118
        """
119
    
120
    def pack(self, obj):
121
        """Pack object with numpy support."""
122

123
class Unpacker:
124
    """
125
    Msgpack unpacker with automatic numpy decoding support.
126
    Inherits from msgpack.Unpacker with numpy-aware object hook.
127
    """
128
    def __init__(self, file_like=None, object_hook=None, **kwargs):
129
        """
130
        Initialize unpacker with numpy support.
131
        
132
        Parameters:
133
        - file_like: Optional file-like object to read from
134
        - object_hook: Optional additional object hook (chained with numpy decoder)
135
        - **kwargs: Additional msgpack.Unpacker parameters (version-dependent)
136
        """
137
```
138

139
### Pack and Unpack Functions
140

141
High-level functions for packing and unpacking with numpy support.
142

143
```python { .api }
144
def pack(o, stream, **kwargs):
145
    """
146
    Pack an object and write it to a stream with numpy support.
147
    
148
    Parameters:
149
    - o: Object to pack
150
    - stream: Output stream to write to
151
    - **kwargs: Additional keyword arguments passed to Packer
152
    """
153

154
def packb(o, **kwargs):
155
    """
156
    Pack an object and return the packed bytes with numpy support.
157
    
158
    Parameters:
159
    - o: Object to pack
160
    - **kwargs: Additional keyword arguments passed to Packer
161
    
162
    Returns:
163
    bytes: Packed bytes
164
    """
165

166
def unpack(stream, **kwargs):
167
    """
168
    Unpack a packed object from a stream with numpy support.
169
    
170
    Parameters:
171
    - stream: Input stream to read from
172
    - **kwargs: Additional keyword arguments passed to underlying unpack function
173
    
174
    Returns:
175
    Unpacked object with numpy types preserved
176
    """
177

178
def unpackb(packed, **kwargs):
179
    """
180
    Unpack a packed object with numpy support.
181
    
182
    Parameters:
183
    - packed: Packed bytes to unpack
184
    - **kwargs: Additional keyword arguments passed to underlying unpackb function
185
    
186
    Returns:
187
    Unpacked object with numpy types preserved
188
    """
189
```
190

191
### Function Aliases
192

193
Convenience aliases for common operations.
194

195
```python { .api }
196
# Aliases for pack/unpack functions
197
load = unpack    # Alias for unpack function
198
loads = unpackb  # Alias for unpackb function  
199
dump = pack      # Alias for pack function
200
dumps = packb    # Alias for packb function
201
```
202

203
### Monkey Patching
204

205
Function to automatically enable numpy support in the msgpack module.
206

207
```python { .api }
208
def patch():
209
    """
210
    Monkey patch msgpack module to enable support for serializing numpy types.
211
    
212
    This function replaces msgpack's Packer, Unpacker, and pack/unpack functions
213
    with numpy-aware versions. After calling this function, all msgpack operations
214
    will automatically handle numpy arrays and scalar types.
215
    """
216
```
217

218
## Supported Data Types
219

220
The library supports serialization/deserialization of:
221

222
- **Numpy arrays**: All dtypes including structured dtypes and object arrays
223
- **Numpy scalars**: All numpy scalar types (bool_, int8, int16, int32, int64, float32, float64, complex64, complex128, etc.)
224
- **Python complex numbers**: Native Python complex type
225
- **Regular Python objects**: Passed through to chain functions or standard msgpack handling
226

227
## Type Preservation
228

229
msgpack-numpy preserves:
230
- Exact numpy data types (dtype information)
231
- Array shapes and dimensions  
232
- Numerical precision
233
- Complex number representations
234

235
Note: Arrays with dtype='O' are serialized using pickle as a fallback, which may have performance implications.
236

237
## Notes
238

239
- Deserialized numpy arrays are read-only and must be copied if modification is needed
240
- The library maintains compatibility with Python 2.7 and 3.5+
241
- Compatible with msgpack versions from <0.4.0 to >=1.0.0
242
- Cross-platform support (Windows, macOS, Linux)