0
# Backend Management
1

2
Backend selection and configuration utilities for optimizing performance based on available libraries and specific requirements. ijson's multi-backend architecture allows it to automatically select the fastest available JSON parsing implementation while maintaining a consistent API.
3

4
## Capabilities
5

6
### Backend Selection
7

8
Get a specific backend by name, useful for forcing a particular implementation or testing different backends.
9

10
```python { .api }
11
def get_backend(backend):
12
    """
13
    Import and return specified backend module.
14
    
15
    Parameters:
16
    - backend (str): Backend name ('yajl2_c', 'yajl2_cffi', 'yajl2', 'yajl', 'python')
17
    
18
    Returns:
19
    Backend module with parsing functions
20
    
21
    Raises:
22
    - ImportError: If backend is not available
23
    """
24
```
25

26
**Usage Examples:**
27

28
```python
29
import ijson
30

31
# Force use of pure Python backend
32
try:
33
    python_backend = ijson.get_backend('python')
34
    items = python_backend.items(json_data, 'data.item')
35
    for item in items:
36
        process(item)
37
except ImportError:
38
    print("Python backend not available")
39

40
# Try fastest backend first
41
for backend_name in ['yajl2_c', 'yajl2_cffi', 'python']:
42
    try:
43
        backend = ijson.get_backend(backend_name)
44
        print(f"Using backend: {backend_name}")
45
        break
46
    except ImportError:
47
        continue
48
```
49

50
### Backend Information
51

52
Access information about the currently selected backend and available backends.
53

54
```python { .api }
55
ALL_BACKENDS: tuple
56
"""
57
All supported backends in descending order of speed.
58
Value: ('yajl2_c', 'yajl2_cffi', 'yajl2', 'yajl', 'python')
59
"""
60

61
backend: object
62
"""
63
Currently selected backend instance.
64
Contains all parsing functions (parse, items, kvitems, etc.)
65
"""
66

67
backend_name: str
68
"""
69
Name of the currently loaded backend.
70
One of: 'yajl2_c', 'yajl2_cffi', 'yajl2', 'yajl', 'python'
71
"""
72
```
73

74
**Usage Examples:**
75

76
```python
77
import ijson
78

79
# Check current backend
80
print(f"Current backend: {ijson.backend_name}")
81
print(f"Available backends: {ijson.ALL_BACKENDS}")
82

83
# Get backend capabilities
84
current_backend = ijson.backend
85
if hasattr(current_backend, 'capabilities'):
86
    caps = current_backend.capabilities
87
    print(f"C-style comments: {caps.c_comments}")
88
    print(f"Multiple values: {caps.multiple_values}")
89
```
90

91
## Backend Types
92

93
### yajl2_c (Fastest)
94

95
C extension using YAJL 2.x library. Provides the best performance but requires compilation during installation.
96

97
```python
98
# Automatically selected if available
99
import ijson
100
print(ijson.backend_name)  # 'yajl2_c'
101

102
# Or force selection
103
backend = ijson.get_backend('yajl2_c')
104
```
105

106
**Characteristics:**
107
- Fastest performance (10-20x faster than Python)
108
- Requires C compiler during installation
109
- Binary wheels available for common platforms
110
- Full feature support including C-style comments
111

112
### yajl2_cffi
113

114
CFFI-based binding to YAJL 2.x library. Good performance without requiring C compiler.
115

116
```python
117
# Force CFFI backend
118
backend = ijson.get_backend('yajl2_cffi')
119
```
120

121
**Characteristics:**
122
- Fast performance (5-15x faster than Python)
123
- No C compiler required (uses CFFI)
124
- Good compatibility across platforms
125
- Full feature support
126

127
### yajl2
128

129
ctypes-based binding to YAJL 2.x library. Moderate performance with maximum compatibility.
130

131
```python
132
# Force ctypes backend
133
backend = ijson.get_backend('yajl2')
134
```
135

136
**Characteristics:**
137
- Moderate performance (3-8x faster than Python)
138
- No compilation required
139
- Works with system-installed YAJL
140
- Full feature support
141

142
### yajl (Legacy)
143

144
ctypes-based binding to YAJL 1.x library. Provided for compatibility with older YAJL installations.
145

146
```python
147
# Force legacy YAJL backend
148
backend = ijson.get_backend('yajl')
149
```
150

151
**Characteristics:**
152
- Moderate performance
153
- Compatible with YAJL 1.x
154
- Limited feature set compared to 2.x versions
155

156
### python (Pure Python)
157

158
Pure Python implementation. Always available but slowest performance.
159

160
```python
161
# Force pure Python backend
162
backend = ijson.get_backend('python')
163
```
164

165
**Characteristics:**
166
- Always available (no dependencies)
167
- Slowest performance but still memory-efficient
168
- Full feature support
169
- Best for debugging and understanding behavior
170

171
## Environment Configuration
172

173
### IJSON_BACKEND Environment Variable
174

175
Force a specific backend using environment variable:
176

177
```bash
178
# Force pure Python backend
179
export IJSON_BACKEND=python
180
python your_script.py
181

182
# Force fastest C backend
183
export IJSON_BACKEND=yajl2_c
184
python your_script.py
185
```
186

187
```python
188
import os
189
import ijson
190

191
# Check if environment override is set
192
if 'IJSON_BACKEND' in os.environ:
193
    print(f"Backend forced to: {os.environ['IJSON_BACKEND']}")
194
    
195
print(f"Using backend: {ijson.backend_name}")
196
```
197

198
### YAJL_DLL Environment Variable
199

200
Specify custom YAJL library location:
201

202
```bash
203
# Use custom YAJL library
204
export YAJL_DLL=/usr/local/lib/libyajl.so.2
205
python your_script.py
206
```
207

208
## Backend Capabilities
209

210
Each backend supports different feature sets through the capabilities system:
211

212
```python { .api }
213
class BackendCapabilities:
214
    """
215
    Capabilities supported by a backend.
216
    """
217
    c_comments: bool              # C-style comments (non-standard JSON)
218
    multiple_values: bool         # Multiple top-level values  
219
    invalid_leading_zeros_detection: bool  # Leading zeros detection
220
    incomplete_json_tokens_detection: bool  # Incomplete token detection
221
    int64: bool                   # 64-bit integer support with use_float=True
222
```
223

224
**Usage Examples:**
225

226
```python
227
import ijson
228

229
# Check backend capabilities
230
caps = ijson.backend.capabilities
231
print(f"Supports C comments: {caps.c_comments}")
232
print(f"Supports multiple values: {caps.multiple_values}")
233
print(f"Detects invalid leading zeros: {caps.invalid_leading_zeros_detection}")
234

235
# Use capabilities to enable features
236
if caps.multiple_values:
237
    # Parse multiple JSON values in stream
238
    items = ijson.parse(stream, multiple_values=True)
239
    
240
if caps.c_comments:
241
    # Parse JSON with C-style comments
242
    data = '{"key": "value" /* comment */}'
243
    result = list(ijson.items(data, ''))
244
```
245

246
## Performance Comparison
247

248
Typical performance characteristics (relative to pure Python):
249

250
| Backend     | Speed Multiplier | Compilation Required | Dependencies |
251
|-------------|------------------|---------------------|--------------|
252
| yajl2_c     | 10-20x          | Yes (C compiler)    | None (wheels available) |
253
| yajl2_cffi  | 5-15x           | No                  | CFFI, YAJL |
254
| yajl2       | 3-8x            | No                  | YAJL 2.x |
255
| yajl        | 3-8x            | No                  | YAJL 1.x |
256
| python      | 1x (baseline)   | No                  | None |
257
258
### Performance Testing
259

260
```python
261
import time
262
import ijson
263

264
def benchmark_backend(backend_name, data, iterations=1000):
265
    try:
266
        backend = ijson.get_backend(backend_name)
267
        start_time = time.time()
268
        
269
        for _ in range(iterations):
270
            list(backend.items(data, 'items.item'))
271
            
272
        elapsed = time.time() - start_time
273
        return elapsed
274
    except ImportError:
275
        return None
276

277
# Benchmark all available backends
278
test_data = '{"items": [' + ','.join([f'{{"id": {i}}}' for i in range(100)]) + ']}'
279

280
results = {}
281
for backend_name in ijson.ALL_BACKENDS:
282
    elapsed = benchmark_backend(backend_name, test_data)
283
    if elapsed is not None:
284
        results[backend_name] = elapsed
285
        print(f"{backend_name}: {elapsed:.4f}s")
286

287
# Find fastest backend
288
if results:
289
    fastest = min(results.keys(), key=lambda k: results[k])
290
    print(f"Fastest available backend: {fastest}")
291
```
292

293
## Error Handling
294

295
### Backend Loading Errors
296

297
```python
298
import ijson
299
from ijson.backends import YAJLImportError
300

301
try:
302
    backend = ijson.get_backend('yajl2_c')
303
except ImportError as e:
304
    print(f"C backend not available: {e}")
305
    # Fall back to pure Python
306
    backend = ijson.get_backend('python')
307

308
try:
309
    backend = ijson.get_backend('yajl2')
310
except YAJLImportError as e:
311
    print(f"YAJL version issue: {e}")
312
```
313

314
### Version Compatibility
315

316
```python
317
import ijson
318

319
# Check for specific backend features
320
backend = ijson.backend
321
if hasattr(backend, 'capabilities'):
322
    if backend.capabilities.int64:
323
        # Safe to use large integers with use_float=True
324
        data = ijson.parse(source, use_float=True)
325
    else:
326
        # Use Decimal for precision
327
        data = ijson.parse(source)
328
```
329

330
## Custom Backend Configuration
331

332
### Backend Selection Strategy
333

334
```python
335
import ijson
336
import os
337

338
def get_optimal_backend():
339
    """Select best backend based on environment and requirements"""
340
    
341
    # Check environment override
342
    if 'IJSON_BACKEND' in os.environ:
343
        return ijson.get_backend(os.environ['IJSON_BACKEND'])
344
    
345
    # Prefer compiled backends for production
346
    if os.environ.get('ENVIRONMENT') == 'production':
347
        for backend_name in ['yajl2_c', 'yajl2_cffi']:
348
            try:
349
                return ijson.get_backend(backend_name)
350
            except ImportError:
351
                continue
352
    
353
    # Use pure Python for development/debugging
354
    if os.environ.get('ENVIRONMENT') == 'development':
355
        return ijson.get_backend('python')
356
    
357
    # Default: use whatever ijson selected
358
    return ijson.backend
359

360
# Use optimal backend
361
optimal_backend = get_optimal_backend()
362
print(f"Selected backend: {optimal_backend.backend_name}")
363
```
364

365
### Feature-Based Selection
366

367
```python
368
import ijson
369

370
def select_backend_for_features(need_comments=False, need_multiple_values=False):
371
    """Select backend based on required features"""
372
    
373
    for backend_name in ijson.ALL_BACKENDS:
374
        try:
375
            backend = ijson.get_backend(backend_name)
376
            caps = backend.capabilities
377
            
378
            if need_comments and not caps.c_comments:
379
                continue
380
            if need_multiple_values and not caps.multiple_values:
381
                continue
382
                
383
            return backend
384
        except ImportError:
385
            continue
386
    
387
    raise RuntimeError("No backend supports required features")
388

389
# Select backend that supports C-style comments
390
backend = select_backend_for_features(need_comments=True)
391
print(f"Using backend with comment support: {backend.backend_name}")
392
```