0
# Zstandard
1

2
Python bindings for the Zstandard compression library, providing high-performance lossless data compression and decompression. This library offers both simple one-shot functions and sophisticated streaming interfaces, supporting advanced features like dictionary compression, multi-threading, and customizable compression parameters while maintaining full compatibility with the standard Zstandard format.
3

4
## Package Information
5

6
- **Package Name**: zstandard
7
- **Language**: Python
8
- **Installation**: `pip install zstandard`
9

10
## Core Imports
11

12
```python
13
import zstandard
14
```
15

16
Common pattern for accessing main functionality:
17

18
```python
19
import zstandard as zstd
20
```
21

22
Import specific components:
23

24
```python
25
from zstandard import ZstdCompressor, ZstdDecompressor, compress, decompress
26
```
27

28
## Basic Usage
29

30
```python
31
import zstandard as zstd
32

33
# Simple one-shot compression and decompression
34
original_data = b"Hello, World! This is some data to compress."
35

36
# Compress data
37
compressed = zstd.compress(original_data, level=3)
38
print(f"Compressed size: {len(compressed)} bytes")
39

40
# Decompress data
41
decompressed = zstd.decompress(compressed)
42
print(f"Decompressed: {decompressed}")
43

44
# Using compressor and decompressor objects for better performance
45
compressor = zstd.ZstdCompressor(level=5)
46
decompressor = zstd.ZstdDecompressor()
47

48
# Multiple compressions with same compressor (more efficient)
49
data1 = b"First piece of data"
50
data2 = b"Second piece of data"
51

52
compressed1 = compressor.compress(data1)
53
compressed2 = compressor.compress(data2)
54

55
decompressed1 = decompressor.decompress(compressed1)
56
decompressed2 = decompressor.decompress(compressed2)
57

58
# File-like interface
59
with zstd.open('data.zst', 'wb') as f:
60
    f.write(b"Data to be compressed and written to file")
61

62
with zstd.open('data.zst', 'rb') as f:
63
    data = f.read()
64
    print(f"Read from file: {data}")
65
```
66

67
## Architecture
68

69
The zstandard library is built around a multi-backend architecture that automatically selects the best available implementation:
70

71
- **C Extension Backend**: High-performance native implementation (default for CPython)
72
- **CFFI Backend**: Pure Python implementation with C bindings (default for PyPy)
73
- **Rust Backend**: Rust-based implementation (experimental)
74

75
Core design patterns:
76

77
- **Compressor/Decompressor Objects**: Reusable contexts for multiple operations
78
- **Streaming Interfaces**: Memory-efficient processing of large data
79
- **Buffer Management**: Advanced buffer types for zero-copy operations
80
- **Multi-threading Support**: Parallel compression/decompression for improved performance
81

82
## Capabilities
83

84
### Simple Compression Operations
85

86
One-shot compression and decompression functions for basic use cases, plus file-like interface for easy integration with existing code.
87

88
```python { .api }
89
def compress(data: bytes, level: int = 3) -> bytes: ...
90
def decompress(data: bytes, max_output_size: int = 0) -> bytes: ...
91
def open(filename, mode: str = "rb", **kwargs): ...
92
```
93

94
[Simple Operations](./simple-operations.md)
95

96
### Advanced Compression
97

98
Full-featured compression with customizable parameters, dictionary support, and streaming interfaces for high-performance applications.
99

100
```python { .api }
101
class ZstdCompressor:
102
    def __init__(self, level: int = 3, **kwargs): ...
103
    def compress(self, data: bytes) -> bytes: ...
104
    def stream_writer(self, writer, **kwargs): ...
105
    def multi_compress_to_buffer(self, data, **kwargs): ...
106

107
class ZstdCompressionParameters:
108
    def __init__(self, **kwargs): ...
109
    @staticmethod
110
    def from_level(level: int, **kwargs): ...
111
```
112

113
[Advanced Compression](./advanced-compression.md)
114

115
### Advanced Decompression
116

117
Sophisticated decompression with streaming support, frame analysis, and batch processing capabilities.
118

119
```python { .api }
120
class ZstdDecompressor:
121
    def __init__(self, **kwargs): ...
122
    def decompress(self, data: bytes, **kwargs) -> bytes: ...
123
    def stream_reader(self, source, **kwargs): ...
124
    def multi_decompress_to_buffer(self, frames, **kwargs): ...
125
```
126

127
[Advanced Decompression](./advanced-decompression.md)
128

129
### Dictionary Compression
130

131
Training and using custom dictionaries for improved compression ratios on similar data.
132

133
```python { .api }
134
class ZstdCompressionDict:
135
    def __init__(self, data: bytes, **kwargs): ...
136
    def dict_id(self) -> int: ...
137
    def precompute_compress(self, **kwargs): ...
138

139
def train_dictionary(dict_size: int, samples: list, **kwargs) -> ZstdCompressionDict: ...
140
```
141

142
[Dictionary Compression](./dictionary-compression.md)
143

144
### Buffer Operations
145

146
Advanced buffer management for zero-copy operations and efficient batch processing.
147

148
```python { .api }
149
class BufferWithSegments:
150
    def __init__(self, data: bytes, segments: bytes): ...
151
    def segments(self): ...
152
    def tobytes(self) -> bytes: ...
153

154
class BufferWithSegmentsCollection:
155
    def __init__(self, *args): ...
156
    def size(self) -> int: ...
157
```
158

159
[Buffer Operations](./buffer-operations.md)
160

161
### Frame Analysis
162

163
Utilities for analyzing zstd frames and extracting metadata without full decompression.
164

165
```python { .api }
166
def frame_content_size(data: bytes) -> int: ...
167
def frame_header_size(data: bytes) -> int: ...
168
def get_frame_parameters(data: bytes) -> FrameParameters: ...
169

170
class FrameParameters:
171
    content_size: int
172
    window_size: int
173
    dict_id: int
174
    has_checksum: bool
175
```
176

177
[Frame Analysis](./frame-analysis.md)
178

179
## Constants and Configuration
180

181
The library exports numerous constants for configuring compression parameters, strategies, and format options:
182

183
```python { .api }
184
# Version information
185
__version__: str
186
backend: str
187
backend_features: set[str]
188
ZSTD_VERSION: tuple
189

190
# Compression levels and strategies
191
MAX_COMPRESSION_LEVEL: int
192
STRATEGY_FAST: int
193
STRATEGY_DFAST: int
194
STRATEGY_GREEDY: int
195
STRATEGY_LAZY: int
196
STRATEGY_LAZY2: int
197
STRATEGY_BTLAZY2: int
198
STRATEGY_BTOPT: int
199
STRATEGY_BTULTRA: int
200
STRATEGY_BTULTRA2: int
201

202
# Recommended buffer sizes
203
COMPRESSION_RECOMMENDED_INPUT_SIZE: int
204
COMPRESSION_RECOMMENDED_OUTPUT_SIZE: int
205
DECOMPRESSION_RECOMMENDED_INPUT_SIZE: int
206
DECOMPRESSION_RECOMMENDED_OUTPUT_SIZE: int
207

208
# Block size limits
209
BLOCKSIZELOG_MAX: int
210
BLOCKSIZE_MAX: int
211

212
# Compression parameter limits
213
WINDOWLOG_MIN: int
214
WINDOWLOG_MAX: int
215
CHAINLOG_MIN: int
216
CHAINLOG_MAX: int
217
HASHLOG_MIN: int
218
HASHLOG_MAX: int
219
MINMATCH_MIN: int
220
MINMATCH_MAX: int
221
SEARCHLOG_MIN: int
222
SEARCHLOG_MAX: int
223
SEARCHLENGTH_MIN: int
224
SEARCHLENGTH_MAX: int
225
TARGETLENGTH_MIN: int
226
TARGETLENGTH_MAX: int
227
LDM_MINMATCH_MIN: int
228
LDM_MINMATCH_MAX: int
229
LDM_BUCKETSIZELOG_MAX: int
230

231
# Format options
232
FORMAT_ZSTD1: int
233
FORMAT_ZSTD1_MAGICLESS: int
234

235
# Dictionary types
236
DICT_TYPE_AUTO: int
237
DICT_TYPE_RAWCONTENT: int
238
DICT_TYPE_FULLDICT: int
239

240
# Flush modes
241
FLUSH_BLOCK: int
242
FLUSH_FRAME: int
243

244
# Compression object flush modes
245
COMPRESSOBJ_FLUSH_FINISH: int
246
COMPRESSOBJ_FLUSH_BLOCK: int
247

248
# Content size indicators
249
CONTENTSIZE_UNKNOWN: int
250
CONTENTSIZE_ERROR: int
251

252
# Frame identification
253
FRAME_HEADER: bytes
254
MAGIC_NUMBER: int
255
```
256

257
## Exception Handling
258

259
```python { .api }
260
class ZstdError(Exception):
261
    """Base exception for zstandard-related errors."""
262
```
263

264
All zstandard operations may raise `ZstdError` or its subclasses for compression/decompression failures, invalid parameters, or corrupted data.