0
# Frame Format
1
2
Frame format compression provides production-ready LZ4 compression with interoperability guarantees, metadata support, and extensive configuration options. This is the recommended interface for most compression use cases.
3
4
## Capabilities
5
6
### Simple Compression Functions
7
8
High-level functions for straightforward compression and decompression operations.
9
10
```python { .api }
11
def compress(data, **kwargs) -> bytes:
12
"""
13
Compress data using LZ4 frame format.
14
15
Args:
16
data (bytes-like): Data to compress
17
block_size (int, optional): Maximum block size (default: BLOCKSIZE_DEFAULT)
18
block_linked (bool, optional): Use block-linked compression (default: True)
19
compression_level (int, optional): Compression level 0-16 (default: 0)
20
content_checksum (bool, optional): Enable content checksum (default: False)
21
block_checksum (bool, optional): Enable block checksum (default: False, requires LZ4 >= 1.8.0)
22
auto_flush (bool, optional): Disable buffering (default: False)
23
source_size (int, optional): Size hint for uncompressed data (default: 0)
24
return_bytearray (bool, optional): Return bytearray instead of bytes (default: False)
25
26
Returns:
27
bytes: Compressed data in LZ4 frame format
28
"""
29
30
def decompress(data, **kwargs) -> bytes:
31
"""
32
Decompress LZ4 frame format data.
33
34
Args:
35
data (bytes-like): Compressed data in LZ4 frame format
36
return_bytearray (bool, optional): Return bytearray instead of bytes (default: False)
37
38
Returns:
39
bytes: Decompressed data
40
"""
41
42
def open(filename, mode, **kwargs) -> LZ4FrameFile:
43
"""
44
Open an LZ4 compressed file.
45
46
Args:
47
filename (str|bytes|PathLike|file): File path or file object
48
mode (str): File mode ('r', 'w', 'x', 'a', with optional 'b' or 't')
49
**kwargs: All LZ4FrameCompressor parameters for writing modes
50
source_size (int, optional): Total uncompressed size hint (default: 0)
51
return_bytearray (bool, optional): Return bytearray objects (default: False)
52
53
Returns:
54
LZ4FrameFile: File-like object for LZ4 compressed files
55
"""
56
```
57
58
### Low-level Context Management
59
60
Functions for manual frame compression with full control over the compression process.
61
62
```python { .api }
63
def create_compression_context():
64
"""Create compression context for low-level operations."""
65
66
def compress_begin(context, **kwargs) -> bytes:
67
"""
68
Begin compression frame with specified options.
69
70
Args:
71
context: Compression context from create_compression_context()
72
**kwargs: Same parameters as compress() function
73
74
Returns:
75
bytes: Frame header
76
"""
77
78
def compress_chunk(context, data, **kwargs) -> bytes:
79
"""
80
Compress data chunk using existing context.
81
82
Args:
83
context: Active compression context
84
data (bytes-like): Data chunk to compress
85
**kwargs: Compression options
86
87
Returns:
88
bytes: Compressed chunk data
89
"""
90
91
def compress_flush(context, **kwargs) -> bytes:
92
"""
93
Flush compression context and finalize frame.
94
95
Args:
96
context: Active compression context
97
**kwargs: Flush options
98
99
Returns:
100
bytes: Final frame data and footer
101
"""
102
103
def create_decompression_context():
104
"""Create decompression context for low-level operations."""
105
106
def reset_decompression_context(context) -> None:
107
"""
108
Reset decompression context (requires LZ4 >= 1.8.0).
109
110
Args:
111
context: Decompression context to reset
112
"""
113
114
def decompress_chunk(context, data, **kwargs) -> tuple[bytes, int, bool]:
115
"""
116
Decompress data chunk using existing context.
117
118
Args:
119
context: Active decompression context
120
data (bytes-like): Compressed data chunk
121
**kwargs: Decompression options
122
123
Returns:
124
tuple: (decompressed_data, bytes_consumed, frame_complete)
125
"""
126
127
def get_frame_info(data, **kwargs) -> dict:
128
"""
129
Get information about LZ4 frame.
130
131
Args:
132
data (bytes-like): Frame data (at least frame header)
133
**kwargs: Options for frame info extraction
134
135
Returns:
136
dict: Frame information including block size, checksums, etc.
137
"""
138
```
139
140
### Constants
141
142
Block size and compression level constants for configuration.
143
144
```python { .api }
145
BLOCKSIZE_DEFAULT: int # Default block size (equals BLOCKSIZE_MAX64KB)
146
BLOCKSIZE_MAX64KB: int # 64 KB maximum block size
147
BLOCKSIZE_MAX256KB: int # 256 KB maximum block size
148
BLOCKSIZE_MAX1MB: int # 1 MB maximum block size
149
BLOCKSIZE_MAX4MB: int # 4 MB maximum block size
150
151
COMPRESSIONLEVEL_MIN: int # Minimum compression level (0)
152
COMPRESSIONLEVEL_MINHC: int # Minimum high-compression level (3)
153
COMPRESSIONLEVEL_MAX: int # Maximum compression level (16)
154
```
155
156
### Incremental Compression
157
158
Class for incremental compression with full configuration control.
159
160
```python { .api }
161
class LZ4FrameCompressor:
162
"""Incremental frame compression with extensive configuration options."""
163
164
def __init__(
165
self,
166
block_size: int = BLOCKSIZE_DEFAULT,
167
block_linked: bool = True,
168
compression_level: int = 0,
169
content_checksum: bool = False,
170
block_checksum: bool = False,
171
auto_flush: bool = False,
172
return_bytearray: bool = False
173
):
174
"""
175
Initialize incremental compressor.
176
177
Args:
178
block_size: Maximum block size for compression
179
block_linked: Use block-linked compression for better ratio
180
compression_level: Compression level 0-16 (0=fastest, 16=best)
181
content_checksum: Enable content checksum verification
182
block_checksum: Enable per-block checksums (requires LZ4 >= 1.8.0)
183
auto_flush: Disable buffering for immediate output
184
return_bytearray: Return bytearray instead of bytes
185
"""
186
187
def begin(self, source_size: int = 0) -> bytes:
188
"""
189
Begin compression frame.
190
191
Args:
192
source_size: Size hint for total uncompressed data
193
194
Returns:
195
bytes: Frame header
196
"""
197
198
def compress(self, data: bytes) -> bytes:
199
"""
200
Compress data chunk.
201
202
Args:
203
data: Data chunk to compress
204
205
Returns:
206
bytes: Compressed chunk data
207
"""
208
209
def flush(self) -> bytes:
210
"""
211
Finish compression and return final frame data.
212
213
Returns:
214
bytes: Final compressed data and frame footer
215
"""
216
217
def reset(self) -> None:
218
"""Reset compressor state for reuse."""
219
220
def has_context(self) -> bool:
221
"""Check if compression context exists."""
222
223
def started(self) -> bool:
224
"""Check if compression frame has been started."""
225
226
def __enter__(self):
227
"""Context manager entry."""
228
return self
229
230
def __exit__(self, exc_type, exc_val, exc_tb):
231
"""Context manager exit."""
232
```
233
234
### Incremental Decompression
235
236
Class for incremental decompression with state management.
237
238
```python { .api }
239
class LZ4FrameDecompressor:
240
"""Incremental frame decompression with state tracking."""
241
242
def __init__(self, return_bytearray: bool = False):
243
"""
244
Initialize incremental decompressor.
245
246
Args:
247
return_bytearray: Return bytearray instead of bytes
248
"""
249
250
@property
251
def eof(self) -> bool:
252
"""True if end-of-stream reached."""
253
254
@property
255
def unused_data(self) -> bytes:
256
"""Data after compressed stream."""
257
258
@property
259
def needs_input(self) -> bool:
260
"""True if more input data needed for decompression."""
261
262
def decompress(self, data: bytes, max_length: int = -1) -> bytes:
263
"""
264
Decompress data chunk.
265
266
Args:
267
data: Compressed data chunk
268
max_length: Maximum bytes to decompress (-1 for unlimited)
269
270
Returns:
271
bytes: Decompressed data
272
"""
273
274
def reset(self) -> None:
275
"""Reset decompressor state for reuse."""
276
277
def __enter__(self):
278
"""Context manager entry."""
279
return self
280
281
def __exit__(self, exc_type, exc_val, exc_tb):
282
"""Context manager exit."""
283
```
284
285
### File Interface
286
287
File-like object providing standard Python file interface for LZ4 compressed files.
288
289
```python { .api }
290
class LZ4FrameFile:
291
"""File-like object for LZ4 compressed files with standard Python file interface."""
292
293
def __init__(
294
self,
295
filename,
296
mode: str,
297
block_size: int = BLOCKSIZE_DEFAULT,
298
block_linked: bool = True,
299
compression_level: int = 0,
300
content_checksum: bool = False,
301
block_checksum: bool = False,
302
auto_flush: bool = False,
303
source_size: int = 0,
304
return_bytearray: bool = False
305
):
306
"""
307
Initialize LZ4 file object.
308
309
Args:
310
filename: File path or file object
311
mode: File mode ('r', 'w', 'x', 'a', with optional 'b' or 't')
312
block_size: Block size for compression
313
block_linked: Use block-linked compression
314
compression_level: Compression level 0-16
315
content_checksum: Enable content checksum
316
block_checksum: Enable block checksums
317
auto_flush: Disable buffering
318
source_size: Total uncompressed size hint
319
return_bytearray: Return bytearray objects
320
"""
321
322
@property
323
def closed(self) -> bool:
324
"""True if file is closed."""
325
326
def read(self, size: int = -1) -> bytes:
327
"""Read uncompressed data."""
328
329
def read1(self, size: int = -1) -> bytes:
330
"""Read data avoiding multiple reads."""
331
332
def readall(self) -> bytes:
333
"""Read entire file."""
334
335
def readline(self, size: int = -1) -> bytes:
336
"""Read line from file."""
337
338
def write(self, data: bytes) -> int:
339
"""Write uncompressed data."""
340
341
def flush(self) -> None:
342
"""Flush write buffers."""
343
344
def seek(self, offset: int, whence: int = 0) -> int:
345
"""Seek to position in file."""
346
347
def tell(self) -> int:
348
"""Get current file position."""
349
350
def peek(self, size: int = -1) -> bytes:
351
"""Peek at buffered data without consuming it."""
352
353
def close(self) -> None:
354
"""Close file and flush buffers."""
355
356
def readable(self) -> bool:
357
"""Check if file is readable."""
358
359
def writable(self) -> bool:
360
"""Check if file is writable."""
361
362
def seekable(self) -> bool:
363
"""Check if file supports seeking."""
364
365
def fileno(self) -> int:
366
"""Get underlying file descriptor."""
367
```
368
369
## Usage Examples
370
371
### Simple Compression
372
373
```python
374
import lz4.frame
375
376
# Basic compression
377
data = b"Hello, World!" * 1000
378
compressed = lz4.frame.compress(data)
379
decompressed = lz4.frame.decompress(compressed)
380
381
# High compression with checksums
382
compressed = lz4.frame.compress(
383
data,
384
compression_level=9,
385
content_checksum=True
386
)
387
```
388
389
### Incremental Processing
390
391
```python
392
import lz4.frame
393
394
# Incremental compression
395
with lz4.frame.LZ4FrameCompressor(compression_level=9) as compressor:
396
header = compressor.begin(source_size=len(total_data))
397
compressed_parts = [header]
398
399
for chunk in data_chunks:
400
compressed_parts.append(compressor.compress(chunk))
401
402
compressed_parts.append(compressor.flush())
403
404
compressed_data = b''.join(compressed_parts)
405
406
# Incremental decompression
407
with lz4.frame.LZ4FrameDecompressor() as decompressor:
408
decompressed_parts = []
409
410
for chunk in compressed_chunks:
411
part = decompressor.decompress(chunk)
412
if part:
413
decompressed_parts.append(part)
414
415
if decompressor.eof:
416
break
417
418
decompressed_data = b''.join(decompressed_parts)
419
```
420
421
### File Operations
422
423
```python
424
import lz4.frame
425
426
# Writing compressed files
427
with lz4.frame.open('data.lz4', 'wb', compression_level=9) as f:
428
f.write(b"Large amounts of data...")
429
f.write(b"More data...")
430
431
# Reading compressed files
432
with lz4.frame.open('data.lz4', 'rb') as f:
433
content = f.read()
434
435
# Text mode with encoding
436
with lz4.frame.open('text.lz4', 'wt', encoding='utf-8') as f:
437
f.write("Unicode text content")
438
439
with lz4.frame.open('text.lz4', 'rt', encoding='utf-8') as f:
440
text = f.read()
441
```