0
# Zstandard
1
2
Python bindings for the Zstandard compression library, providing high-performance lossless data compression and decompression. This library offers both simple one-shot functions and sophisticated streaming interfaces, supporting advanced features like dictionary compression, multi-threading, and customizable compression parameters while maintaining full compatibility with the standard Zstandard format.
3
4
## Package Information
5
6
- **Package Name**: zstandard
7
- **Language**: Python
8
- **Installation**: `pip install zstandard`
9
10
## Core Imports
11
12
```python
13
import zstandard
14
```
15
16
Common pattern for accessing main functionality:
17
18
```python
19
import zstandard as zstd
20
```
21
22
Import specific components:
23
24
```python
25
from zstandard import ZstdCompressor, ZstdDecompressor, compress, decompress
26
```
27
28
## Basic Usage
29
30
```python
31
import zstandard as zstd
32
33
# Simple one-shot compression and decompression
34
original_data = b"Hello, World! This is some data to compress."
35
36
# Compress data
37
compressed = zstd.compress(original_data, level=3)
38
print(f"Compressed size: {len(compressed)} bytes")
39
40
# Decompress data
41
decompressed = zstd.decompress(compressed)
42
print(f"Decompressed: {decompressed}")
43
44
# Using compressor and decompressor objects for better performance
45
compressor = zstd.ZstdCompressor(level=5)
46
decompressor = zstd.ZstdDecompressor()
47
48
# Multiple compressions with same compressor (more efficient)
49
data1 = b"First piece of data"
50
data2 = b"Second piece of data"
51
52
compressed1 = compressor.compress(data1)
53
compressed2 = compressor.compress(data2)
54
55
decompressed1 = decompressor.decompress(compressed1)
56
decompressed2 = decompressor.decompress(compressed2)
57
58
# File-like interface
59
with zstd.open('data.zst', 'wb') as f:
60
f.write(b"Data to be compressed and written to file")
61
62
with zstd.open('data.zst', 'rb') as f:
63
data = f.read()
64
print(f"Read from file: {data}")
65
```
66
67
## Architecture
68
69
The zstandard library is built around a multi-backend architecture that automatically selects the best available implementation:
70
71
- **C Extension Backend**: High-performance native implementation (default for CPython)
72
- **CFFI Backend**: Pure Python implementation with C bindings (default for PyPy)
73
- **Rust Backend**: Rust-based implementation (experimental)
74
75
Core design patterns:
76
77
- **Compressor/Decompressor Objects**: Reusable contexts for multiple operations
78
- **Streaming Interfaces**: Memory-efficient processing of large data
79
- **Buffer Management**: Advanced buffer types for zero-copy operations
80
- **Multi-threading Support**: Parallel compression/decompression for improved performance
81
82
## Capabilities
83
84
### Simple Compression Operations
85
86
One-shot compression and decompression functions for basic use cases, plus file-like interface for easy integration with existing code.
87
88
```python { .api }
89
def compress(data: bytes, level: int = 3) -> bytes: ...
90
def decompress(data: bytes, max_output_size: int = 0) -> bytes: ...
91
def open(filename, mode: str = "rb", **kwargs): ...
92
```
93
94
[Simple Operations](./simple-operations.md)
95
96
### Advanced Compression
97
98
Full-featured compression with customizable parameters, dictionary support, and streaming interfaces for high-performance applications.
99
100
```python { .api }
101
class ZstdCompressor:
102
def __init__(self, level: int = 3, **kwargs): ...
103
def compress(self, data: bytes) -> bytes: ...
104
def stream_writer(self, writer, **kwargs): ...
105
def multi_compress_to_buffer(self, data, **kwargs): ...
106
107
class ZstdCompressionParameters:
108
def __init__(self, **kwargs): ...
109
@staticmethod
110
def from_level(level: int, **kwargs): ...
111
```
112
113
[Advanced Compression](./advanced-compression.md)
114
115
### Advanced Decompression
116
117
Sophisticated decompression with streaming support, frame analysis, and batch processing capabilities.
118
119
```python { .api }
120
class ZstdDecompressor:
121
def __init__(self, **kwargs): ...
122
def decompress(self, data: bytes, **kwargs) -> bytes: ...
123
def stream_reader(self, source, **kwargs): ...
124
def multi_decompress_to_buffer(self, frames, **kwargs): ...
125
```
126
127
[Advanced Decompression](./advanced-decompression.md)
128
129
### Dictionary Compression
130
131
Training and using custom dictionaries for improved compression ratios on similar data.
132
133
```python { .api }
134
class ZstdCompressionDict:
135
def __init__(self, data: bytes, **kwargs): ...
136
def dict_id(self) -> int: ...
137
def precompute_compress(self, **kwargs): ...
138
139
def train_dictionary(dict_size: int, samples: list, **kwargs) -> ZstdCompressionDict: ...
140
```
141
142
[Dictionary Compression](./dictionary-compression.md)
143
144
### Buffer Operations
145
146
Advanced buffer management for zero-copy operations and efficient batch processing.
147
148
```python { .api }
149
class BufferWithSegments:
150
def __init__(self, data: bytes, segments: bytes): ...
151
def segments(self): ...
152
def tobytes(self) -> bytes: ...
153
154
class BufferWithSegmentsCollection:
155
def __init__(self, *args): ...
156
def size(self) -> int: ...
157
```
158
159
[Buffer Operations](./buffer-operations.md)
160
161
### Frame Analysis
162
163
Utilities for analyzing zstd frames and extracting metadata without full decompression.
164
165
```python { .api }
166
def frame_content_size(data: bytes) -> int: ...
167
def frame_header_size(data: bytes) -> int: ...
168
def get_frame_parameters(data: bytes) -> FrameParameters: ...
169
170
class FrameParameters:
171
content_size: int
172
window_size: int
173
dict_id: int
174
has_checksum: bool
175
```
176
177
[Frame Analysis](./frame-analysis.md)
178
179
## Constants and Configuration
180
181
The library exports numerous constants for configuring compression parameters, strategies, and format options:
182
183
```python { .api }
184
# Version information
185
__version__: str
186
backend: str
187
backend_features: set[str]
188
ZSTD_VERSION: tuple
189
190
# Compression levels and strategies
191
MAX_COMPRESSION_LEVEL: int
192
STRATEGY_FAST: int
193
STRATEGY_DFAST: int
194
STRATEGY_GREEDY: int
195
STRATEGY_LAZY: int
196
STRATEGY_LAZY2: int
197
STRATEGY_BTLAZY2: int
198
STRATEGY_BTOPT: int
199
STRATEGY_BTULTRA: int
200
STRATEGY_BTULTRA2: int
201
202
# Recommended buffer sizes
203
COMPRESSION_RECOMMENDED_INPUT_SIZE: int
204
COMPRESSION_RECOMMENDED_OUTPUT_SIZE: int
205
DECOMPRESSION_RECOMMENDED_INPUT_SIZE: int
206
DECOMPRESSION_RECOMMENDED_OUTPUT_SIZE: int
207
208
# Block size limits
209
BLOCKSIZELOG_MAX: int
210
BLOCKSIZE_MAX: int
211
212
# Compression parameter limits
213
WINDOWLOG_MIN: int
214
WINDOWLOG_MAX: int
215
CHAINLOG_MIN: int
216
CHAINLOG_MAX: int
217
HASHLOG_MIN: int
218
HASHLOG_MAX: int
219
MINMATCH_MIN: int
220
MINMATCH_MAX: int
221
SEARCHLOG_MIN: int
222
SEARCHLOG_MAX: int
223
SEARCHLENGTH_MIN: int
224
SEARCHLENGTH_MAX: int
225
TARGETLENGTH_MIN: int
226
TARGETLENGTH_MAX: int
227
LDM_MINMATCH_MIN: int
228
LDM_MINMATCH_MAX: int
229
LDM_BUCKETSIZELOG_MAX: int
230
231
# Format options
232
FORMAT_ZSTD1: int
233
FORMAT_ZSTD1_MAGICLESS: int
234
235
# Dictionary types
236
DICT_TYPE_AUTO: int
237
DICT_TYPE_RAWCONTENT: int
238
DICT_TYPE_FULLDICT: int
239
240
# Flush modes
241
FLUSH_BLOCK: int
242
FLUSH_FRAME: int
243
244
# Compression object flush modes
245
COMPRESSOBJ_FLUSH_FINISH: int
246
COMPRESSOBJ_FLUSH_BLOCK: int
247
248
# Content size indicators
249
CONTENTSIZE_UNKNOWN: int
250
CONTENTSIZE_ERROR: int
251
252
# Frame identification
253
FRAME_HEADER: bytes
254
MAGIC_NUMBER: int
255
```
256
257
## Exception Handling
258
259
```python { .api }
260
class ZstdError(Exception):
261
"""Base exception for zstandard-related errors."""
262
```
263
264
All zstandard operations may raise `ZstdError` or its subclasses for compression/decompression failures, invalid parameters, or corrupted data.