0
# Cache Backends
1
2
Cache backends provide persistent storage for HTTP responses with different performance characteristics, scalability options, and deployment requirements. Requests-cache supports multiple backends from simple in-memory caching to distributed storage solutions.
3
4
## Capabilities
5
6
### Backend Initialization
7
8
Central function for initializing any backend by name, class, or instance.
9
10
```python { .api }
11
def init_backend(
12
cache_name: StrOrPath,
13
backend: Optional[BackendSpecifier] = None,
14
**kwargs
15
) -> BaseCache:
16
"""
17
Initialize a backend from name, class, or instance.
18
19
Parameters:
20
- cache_name: Cache identifier (path, namespace, etc.)
21
- backend: Backend name or instance
22
- **kwargs: Backend-specific configuration options
23
24
Returns:
25
Initialized backend instance
26
27
Available backend names:
28
'sqlite', 'redis', 'mongodb', 'gridfs', 'dynamodb', 'filesystem', 'memory'
29
"""
30
```
31
32
### SQLite Backend
33
34
File-based SQLite database backend offering good performance for single-process applications with optional WAL mode for better concurrency.
35
36
```python { .api }
37
class SQLiteCache(BaseCache):
38
def __init__(
39
self,
40
db_path: StrOrPath = 'http_cache.sqlite',
41
use_cache_dir: bool = False,
42
use_temp: bool = False,
43
use_memory: bool = False,
44
busy_timeout: float = 30.0,
45
fast_save: bool = False,
46
wal: bool = False,
47
**kwargs
48
):
49
"""
50
SQLite-based cache backend.
51
52
Parameters:
53
- db_path: Database file path
54
- use_cache_dir: Store in user cache directory
55
- use_temp: Store in temporary directory
56
- use_memory: Use in-memory database (":memory:")
57
- busy_timeout: Seconds to wait for database locks
58
- fast_save: Skip fsync for better performance
59
- wal: Enable Write-Ahead Logging for concurrency
60
"""
61
62
class SQLiteDict(BaseStorage):
63
"""SQLite storage implementation for key-value operations."""
64
```
65
66
#### Usage Examples
67
68
```python
69
from requests_cache import CachedSession
70
71
# Default SQLite backend
72
session = CachedSession('my_cache') # Creates http_cache.sqlite
73
74
# Custom SQLite configuration
75
session = CachedSession(
76
cache_name='custom_cache.db',
77
backend='sqlite',
78
use_cache_dir=True, # Store in user cache directory
79
wal=True, # Enable WAL mode
80
busy_timeout=60.0 # Wait up to 60 seconds for locks
81
)
82
83
# In-memory SQLite (lost on exit)
84
session = CachedSession(
85
backend='sqlite',
86
use_memory=True
87
)
88
```
89
90
### Redis Backend
91
92
Redis-based backend for distributed caching with high performance and scalability options. Supports Redis Cluster and multiple data structures.
93
94
```python { .api }
95
class RedisCache(BaseCache):
96
def __init__(
97
self,
98
namespace: str = 'http_cache',
99
connection: Optional[Redis] = None,
100
**kwargs
101
):
102
"""
103
Redis-based cache backend.
104
105
Parameters:
106
- namespace: Redis key prefix
107
- connection: Existing Redis connection
108
- **kwargs: Redis connection parameters (host, port, db, etc.)
109
"""
110
111
class RedisDict(BaseStorage):
112
"""Redis storage using separate keys for each cached response."""
113
114
class RedisHashDict(BaseStorage):
115
"""Redis storage using hash data structure for better memory efficiency."""
116
```
117
118
#### Usage Examples
119
120
```python
121
from requests_cache import CachedSession
122
import redis
123
124
# Default Redis connection (localhost:6379)
125
session = CachedSession(
126
cache_name='my_cache',
127
backend='redis'
128
)
129
130
# Custom Redis configuration
131
session = CachedSession(
132
cache_name='api_cache',
133
backend='redis',
134
host='redis.example.com',
135
port=6380,
136
db=1,
137
password='secret'
138
)
139
140
# Use existing Redis connection
141
redis_client = redis.Redis(host='localhost', port=6379, db=0)
142
session = CachedSession(
143
cache_name='shared_cache',
144
backend='redis',
145
connection=redis_client
146
)
147
```
148
149
### MongoDB Backends
150
151
MongoDB-based backends supporting both standard collections and GridFS for large responses.
152
153
```python { .api }
154
class MongoCache(BaseCache):
155
def __init__(
156
self,
157
db_name: str = 'http_cache',
158
connection: Optional[MongoClient] = None,
159
**kwargs
160
):
161
"""
162
MongoDB-based cache backend.
163
164
Parameters:
165
- db_name: MongoDB database name
166
- connection: Existing MongoClient connection
167
- **kwargs: MongoDB connection parameters
168
"""
169
170
class MongoDict(BaseStorage):
171
"""MongoDB storage using standard collections."""
172
173
class GridFSCache(BaseCache):
174
def __init__(
175
self,
176
db_name: str = 'http_cache',
177
connection: Optional[MongoClient] = None,
178
**kwargs
179
):
180
"""
181
GridFS-based cache backend for large responses.
182
183
Parameters:
184
- db_name: MongoDB database name
185
- connection: Existing MongoClient connection
186
- **kwargs: MongoDB connection parameters
187
"""
188
189
class GridFSDict(BaseStorage):
190
"""GridFS storage for handling large response bodies."""
191
```
192
193
#### Usage Examples
194
195
```python
196
from requests_cache import CachedSession
197
from pymongo import MongoClient
198
199
# Default MongoDB connection
200
session = CachedSession(
201
cache_name='web_cache',
202
backend='mongodb'
203
)
204
205
# Custom MongoDB configuration
206
session = CachedSession(
207
cache_name='api_cache',
208
backend='mongodb',
209
host='mongo.example.com',
210
port=27018,
211
username='cache_user',
212
password='password'
213
)
214
215
# GridFS for large responses
216
session = CachedSession(
217
cache_name='media_cache',
218
backend='gridfs'
219
)
220
221
# Use existing MongoDB connection
222
client = MongoClient('mongodb://localhost:27017/')
223
session = CachedSession(
224
cache_name='shared_cache',
225
backend='mongodb',
226
connection=client
227
)
228
```
229
230
### DynamoDB Backend
231
232
AWS DynamoDB backend for serverless and cloud-native applications with automatic scaling and global distribution.
233
234
```python { .api }
235
class DynamoDbCache(BaseCache):
236
def __init__(
237
self,
238
table_name: str = 'http_cache',
239
namespace: str = 'default',
240
region_name: str = 'us-east-1',
241
**kwargs
242
):
243
"""
244
DynamoDB-based cache backend.
245
246
Parameters:
247
- table_name: DynamoDB table name
248
- namespace: Partition key prefix
249
- region_name: AWS region
250
- **kwargs: Additional boto3 configuration
251
"""
252
253
class DynamoDbDict(BaseStorage):
254
"""DynamoDB storage implementation."""
255
```
256
257
#### Usage Examples
258
259
```python
260
from requests_cache import CachedSession
261
262
# Default DynamoDB configuration
263
session = CachedSession(
264
cache_name='api_cache',
265
backend='dynamodb',
266
table_name='my-cache-table'
267
)
268
269
# Custom AWS configuration
270
session = CachedSession(
271
cache_name='global_cache',
272
backend='dynamodb',
273
table_name='global-api-cache',
274
region_name='us-west-2',
275
aws_access_key_id='ACCESS_KEY',
276
aws_secret_access_key='SECRET_KEY'
277
)
278
```
279
280
### Filesystem Backend
281
282
File-system based backend storing responses as individual files, useful for sharing cache between processes and debugging.
283
284
```python { .api }
285
class FileCache(BaseCache):
286
def __init__(
287
self,
288
cache_name: StrOrPath = 'http_cache',
289
use_cache_dir: bool = False,
290
use_temp: bool = False,
291
**kwargs
292
):
293
"""
294
Filesystem-based cache backend.
295
296
Parameters:
297
- cache_name: Cache directory path
298
- use_cache_dir: Store in user cache directory
299
- use_temp: Store in temporary directory
300
"""
301
302
class FileDict(BaseStorage):
303
"""Filesystem storage using individual files."""
304
```
305
306
#### Usage Examples
307
308
```python
309
from requests_cache import CachedSession
310
311
# Default filesystem backend
312
session = CachedSession(
313
cache_name='my_cache',
314
backend='filesystem' # Creates 'my_cache' directory
315
)
316
317
# Custom filesystem configuration
318
session = CachedSession(
319
cache_name='/path/to/cache',
320
backend='filesystem',
321
use_cache_dir=True # Use system cache directory
322
)
323
```
324
325
### Memory Backend
326
327
In-memory backend using Python dictionaries. Fast but non-persistent, suitable for temporary caching within single process.
328
329
```python { .api }
330
class BaseCache:
331
"""In-memory cache backend (when used with backend='memory')."""
332
```
333
334
#### Usage Examples
335
336
```python
337
from requests_cache import CachedSession
338
339
# In-memory caching (lost when process exits)
340
session = CachedSession(
341
cache_name='temp_cache',
342
backend='memory'
343
)
344
```
345
346
### Backend Selection
347
348
Guidance for choosing appropriate backends:
349
350
- **SQLite**: Default choice, good for single-process applications
351
- **Redis**: Best for distributed applications, high performance
352
- **MongoDB/GridFS**: Good for complex queries, GridFS for large responses
353
- **DynamoDB**: Serverless applications, automatic scaling
354
- **Filesystem**: Debugging, sharing between processes
355
- **Memory**: Temporary caching, testing
356
357
## Base Classes
358
359
Foundation classes that all backends inherit from:
360
361
```python { .api }
362
class BaseCache:
363
"""Base class for all cache backends."""
364
365
def clear(self) -> None:
366
"""Clear all cached responses."""
367
368
def delete(self, *args, **kwargs) -> int:
369
"""Delete responses matching conditions."""
370
371
def get_response(self, key: str) -> Optional[CachedResponse]:
372
"""Get cached response by key."""
373
374
def save_response(
375
self,
376
response: AnyResponse,
377
cache_key: str,
378
expires: Optional[datetime] = None
379
) -> None:
380
"""Save response to cache."""
381
382
class BaseStorage:
383
"""Base storage interface for key-value operations."""
384
385
class DictStorage(BaseStorage):
386
"""In-memory dict-based storage."""
387
```
388
389
## Types
390
391
```python { .api }
392
# Backend specification types
393
BackendSpecifier = Union[str, BaseCache]
394
StrOrPath = Union[Path, str]
395
396
# Backend name constants
397
BACKEND_CLASSES = {
398
'dynamodb': DynamoDbCache,
399
'filesystem': FileCache,
400
'gridfs': GridFSCache,
401
'memory': BaseCache,
402
'mongodb': MongoCache,
403
'redis': RedisCache,
404
'sqlite': SQLiteCache,
405
}
406
```