0
# High-Performance Formatting
1

2
Alternative JSON formatters using orjson and msgspec libraries for improved performance in high-throughput logging scenarios. These formatters provide significant speed improvements over the standard json module.
3

4
## Core Imports
5

6
For orjson formatter:
7

8
```python
9
from pythonjsonlogger.orjson import OrjsonFormatter, orjson_default
10
```
11

12
For msgspec formatter:
13

14
```python
15
from pythonjsonlogger.msgspec import MsgspecFormatter, msgspec_default
16
```
17

18
Checking availability:
19

20
```python
21
import pythonjsonlogger
22
if pythonjsonlogger.ORJSON_AVAILABLE:
23
    # orjson is available
24
if pythonjsonlogger.MSGSPEC_AVAILABLE:
25
    # msgspec is available
26
```
27

28
## Capabilities
29

30
### OrjsonFormatter Class
31

32
JSON formatter using the orjson library for high-performance serialization.
33

34
```python { .api }
35
class OrjsonFormatter(BaseJsonFormatter):
36
    def __init__(
37
        self,
38
        *args,
39
        json_default: OptionalCallableOrStr = orjson_default,
40
        json_indent: bool = False,
41
        **kwargs,
42
    ) -> None:
43
        """
44
        orjson-based JSON formatter initialization.
45

46
        Parameters:
47
        - args: see BaseJsonFormatter
48
        - json_default: function for encoding non-standard objects (defaults to orjson_default)
49
        - json_indent: indent output with 2 spaces
50
        - kwargs: see BaseJsonFormatter
51
        """
52

53
    def jsonify_log_record(self, log_record: LogRecord) -> str:
54
        """
55
        Convert log record to JSON string using orjson.
56

57
        Parameters:
58
        - log_record: dictionary containing log data
59

60
        Returns:
61
        JSON string representation using orjson serialization
62
        """
63
```
64

65
### MsgspecFormatter Class
66

67
JSON formatter using the msgspec library for high-performance serialization with minimal memory overhead.
68

69
```python { .api }
70
class MsgspecFormatter(BaseJsonFormatter):
71
    def __init__(
72
        self,
73
        *args,
74
        json_default: OptionalCallableOrStr = msgspec_default,
75
        **kwargs,
76
    ) -> None:
77
        """
78
        msgspec-based JSON formatter initialization.
79

80
        Parameters:
81
        - args: see BaseJsonFormatter  
82
        - json_default: function for encoding non-standard objects (defaults to msgspec_default)
83
        - kwargs: see BaseJsonFormatter
84
        """
85

86
    def jsonify_log_record(self, log_record: LogRecord) -> str:
87
        """
88
        Convert log record to JSON string using msgspec.
89

90
        Parameters:
91
        - log_record: dictionary containing log data
92

93
        Returns:
94
        JSON string representation using msgspec serialization
95
        """
96
```
97

98
### Default Encoding Functions
99

100
Default encoder functions for handling non-standard types with orjson and msgspec.
101

102
```python { .api }
103
def orjson_default(obj: Any) -> Any:
104
    """
105
    orjson default encoder function for non-standard types.
106

107
    Handles exceptions, tracebacks, bytes, enums, type objects,
108
    and unknown types.
109

110
    Parameters:
111
    - obj: object to encode
112

113
    Returns:
114
    Encodable representation of the object
115
    """
116

117
def msgspec_default(obj: Any) -> Any:
118
    """
119
    msgspec default encoder function for non-standard types.
120

121
    Handles exceptions, tracebacks, enums, type objects,
122
    and unknown types.
123

124
    Parameters:
125
    - obj: object to encode
126

127
    Returns:
128
    Encodable representation of the object
129
    """
130
```
131

132
## Usage Examples
133

134
### Using OrjsonFormatter
135

136
```python
137
import logging
138
from pythonjsonlogger.orjson import OrjsonFormatter
139

140
# Check if orjson is available
141
import pythonjsonlogger
142
if pythonjsonlogger.ORJSON_AVAILABLE:
143
    # Set up high-performance JSON logging
144
    logger = logging.getLogger('high_perf_app')
145
    handler = logging.StreamHandler()
146
    
147
    # Use orjson formatter with indentation
148
    formatter = OrjsonFormatter(
149
        '%(levelname)s %(name)s %(message)s',
150
        json_indent=True
151
    )
152
    handler.setFormatter(formatter)
153
    logger.addHandler(handler)
154
    
155
    # High-speed JSON logging
156
    logger.info("Processing complete", extra={"records": 10000, "duration": 2.5})
157
```
158

159
### Using MsgspecFormatter
160

161
```python
162
import logging
163
from pythonjsonlogger.msgspec import MsgspecFormatter
164

165
# Check if msgspec is available
166
import pythonjsonlogger
167
if pythonjsonlogger.MSGSPEC_AVAILABLE:
168
    # Set up msgspec-based JSON logging  
169
    logger = logging.getLogger('msgspec_app')
170
    handler = logging.StreamHandler()
171
    
172
    formatter = MsgspecFormatter('%(levelname)s %(message)s')
173
    handler.setFormatter(formatter)
174
    logger.addHandler(handler)
175
    
176
    # Memory-efficient JSON logging
177
    logger.warning("Memory usage high", extra={"memory_mb": 1024, "threshold": 512})
178
```
179

180
### Custom Default Functions
181

182
```python
183
from pythonjsonlogger.orjson import OrjsonFormatter
184
import uuid
185

186
def custom_orjson_default(obj):
187
    if isinstance(obj, uuid.UUID):
188
        return str(obj)
189
    # Fall back to built-in default handling
190
    from pythonjsonlogger.orjson import orjson_default
191
    return orjson_default(obj)
192

193
formatter = OrjsonFormatter(
194
    '%(levelname)s %(message)s',
195
    json_default=custom_orjson_default
196
)
197
```
198

199
### Conditional Formatter Selection
200

201
```python
202
import logging
203
import pythonjsonlogger
204

205
def get_best_formatter():
206
    """Select the best available JSON formatter."""
207
    if pythonjsonlogger.ORJSON_AVAILABLE:
208
        from pythonjsonlogger.orjson import OrjsonFormatter
209
        return OrjsonFormatter('%(levelname)s %(name)s %(message)s')
210
    elif pythonjsonlogger.MSGSPEC_AVAILABLE:
211
        from pythonjsonlogger.msgspec import MsgspecFormatter
212
        return MsgspecFormatter('%(levelname)s %(name)s %(message)s')
213
    else:
214
        from pythonjsonlogger.json import JsonFormatter
215
        return JsonFormatter('%(levelname)s %(name)s %(message)s')
216

217
# Use the best available formatter
218
logger = logging.getLogger('adaptive_app')
219
handler = logging.StreamHandler()
220
handler.setFormatter(get_best_formatter())
221
logger.addHandler(handler)
222
```
223

224
## Performance Considerations
225

226
### orjson Benefits
227
- Significantly faster JSON serialization than standard library
228
- Better handling of large data structures
229
- Native support for datetime objects
230
- Supports indentation for readable output
231

232
### msgspec Benefits  
233
- Extremely fast serialization with minimal memory allocation
234
- Smallest memory footprint
235
- Excellent for high-frequency logging scenarios
236
- Built-in support for many Python types
237

238
### Installation
239
```bash
240
# For orjson support (not available on PyPy)
241
pip install python-json-logger[orjson]
242

243
# For msgspec support (not available on PyPy)  
244
pip install python-json-logger[msgspec]
245

246
# For both high-performance options
247
pip install python-json-logger[dev]
248
```
249

250
## Type Handling
251

252
Both orjson and msgspec formatters handle these types through their default functions:
253

254
- **Exception objects**: Converted to "ExceptionClass: message" format
255
- **Traceback objects**: Converted to formatted traceback strings  
256
- **Enum objects**: Converted to their values
257
- **bytes/bytearray**: Converted to base64 strings (orjson only)
258
- **type objects**: Converted to class names
259
- **Unknown types**: Fallback to str() or repr(), or "__could_not_encode__"
260

261
Note: orjson and msgspec have native support for datetime objects, so they don't use the defaults module for those types.