0
# Backend Cores
1

2
Cachier supports multiple storage backends to meet different caching requirements, from simple in-memory caching to distributed cross-machine caching solutions. Each backend is optimized for specific use cases and environments.
3

4
## Available Backends
5

6
```python { .api }
7
Backend = Literal["pickle", "mongo", "memory", "redis", "sql"]
8
```
9

10
## Capabilities
11

12
### Pickle Backend (Default)
13

14
File-based persistent caching using Python's pickle serialization:
15

16
```python { .api }
17
@cachier(backend='pickle')
18
def cached_function(args):
19
    return computation(args)
20

21
# With custom cache directory
22
@cachier(
23
    backend='pickle',
24
    cache_dir='/path/to/cache',
25
    pickle_reload=True,
26
    separate_files=False
27
)
28
def file_cached_function(args):
29
    return computation(args)
30
```
31

32
**Pickle Backend Parameters**:
33
- `cache_dir`: Directory for cache files (default: `~/.cachier/`)
34
- `pickle_reload`: Reload cache on each read for thread safety (default: `True`)
35
- `separate_files`: Split cache into separate files per argument set (default: `False`)
36

37
**Use Cases**:
38
- Single-machine applications
39
- Persistent caching across application restarts
40
- Development and testing environments
41
- Functions with complex return types
42

43
**Advantages**:
44
- No external dependencies
45
- Handles any picklable Python object
46
- Persistent across restarts
47
- File-based inspection possible
48

49
**Limitations**:
50
- Single-machine only
51
- Not suitable for high-concurrency scenarios
52
- Pickle security considerations with untrusted data
53

54
### Memory Backend
55

56
In-memory caching for fastest access within a single process:
57

58
```python { .api }
59
@cachier(backend='memory')
60
def memory_cached_function(args):
61
    return computation(args)
62

63
# With calculation timeout
64
@cachier(
65
    backend='memory',
66
    wait_for_calc_timeout=30
67
)
68
def concurrent_memory_function(args):
69
    return computation(args)
70
```
71

72
**Memory Backend Parameters**:
73
- `wait_for_calc_timeout`: Max wait time for ongoing calculations (default: `0` - wait forever)
74

75
**Use Cases**:
76
- High-performance applications requiring fastest access
77
- Temporary caching within application lifecycle
78
- Testing and development
79
- Functions called frequently within short time periods
80

81
**Advantages**:
82
- Fastest access times
83
- No I/O overhead
84
- Thread-safe within process
85
- No external dependencies
86

87
**Limitations**:
88
- Lost on application restart
89
- Memory usage grows with cache size
90
- Single-process only
91

92
### MongoDB Backend
93

94
Distributed caching using MongoDB for cross-machine cache sharing:
95

96
```python { .api }
97
def get_mongo_collection():
98
    import pymongo
99
    client = pymongo.MongoClient('mongodb://localhost:27017/')
100
    return client.cache_db.function_cache
101

102
@cachier(
103
    backend='mongo',
104
    mongetter=get_mongo_collection,
105
    wait_for_calc_timeout=60
106
)
107
def distributed_function(args):
108
    return computation(args)
109
```
110

111
**MongoDB Backend Parameters**:
112
- `mongetter`: Callable returning `pymongo.Collection` with write permissions
113
- `wait_for_calc_timeout`: Max wait time for ongoing calculations (default: `0`)
114

115
**Use Cases**:
116
- Distributed applications across multiple machines
117
- Microservices sharing computation results
118
- Cloud deployments with shared state
119
- Long-running computations with high reuse
120

121
**Advantages**:
122
- Cross-machine cache sharing
123
- Persistent and durable storage
124
- Handles concurrent access well
125
- Scalable with MongoDB clusters
126

127
**Limitations**:
128
- Requires MongoDB infrastructure
129
- Network latency for cache operations
130
- Additional dependency (pymongo)
131
- Serialization limitations
132

133
### Redis Backend
134

135
High-performance distributed caching using Redis:
136

137
```python { .api }
138
def get_redis_client():
139
    import redis
140
    return redis.Redis(host='localhost', port=6379, db=0)
141

142
@cachier(
143
    backend='redis',
144
    redis_client=get_redis_client,
145
    wait_for_calc_timeout=30
146
)
147
def redis_cached_function(args):
148
    return computation(args)
149

150
# Using Redis client instance directly
151
import redis
152
redis_client = redis.Redis(host='cache-server', port=6379)
153

154
@cachier(
155
    backend='redis',
156
    redis_client=redis_client
157
)
158
def fast_distributed_function(args):
159
    return computation(args)
160
```
161

162
**Redis Backend Parameters**:
163
- `redis_client`: Redis client instance or callable returning Redis client
164
- `wait_for_calc_timeout`: Max wait time for ongoing calculations (default: `0`)
165

166
**Use Cases**:
167
- High-performance distributed caching
168
- Real-time applications requiring fast cache access
169
- Session storage and temporary data
170
- Microservices with shared cache layer
171

172
**Advantages**:
173
- Very fast access times
174
- Built-in expiration support
175
- Cross-machine cache sharing
176
- Battle-tested for high concurrency
177

178
**Limitations**:
179
- Requires Redis infrastructure
180
- Memory-based storage (though persistent options available)
181
- Additional dependency (redis-py)
182

183
### SQL Backend
184

185
Database-backed caching using SQLAlchemy for enterprise environments:
186

187
```python { .api }
188
# Using connection string
189
@cachier(
190
    backend='sql',
191
    sql_engine='postgresql://user:pass@localhost/cache_db'
192
)
193
def sql_cached_function(args):
194
    return computation(args)
195

196
# Using SQLAlchemy Engine
197
from sqlalchemy import create_engine
198

199
engine = create_engine('sqlite:///cache.db')
200

201
@cachier(
202
    backend='sql',
203
    sql_engine=engine
204
)
205
def database_cached_function(args):
206
    return computation(args)
207

208
# Using callable for lazy connection
209
def get_sql_engine():
210
    return create_engine('mysql://user:pass@dbserver/cache')
211

212
@cachier(
213
    backend='sql',
214
    sql_engine=get_sql_engine,
215
    wait_for_calc_timeout=45
216
)
217
def enterprise_function(args):
218
    return computation(args)
219
```
220

221
**SQL Backend Parameters**:
222
- `sql_engine`: SQLAlchemy connection string, Engine instance, or callable returning Engine
223
- `wait_for_calc_timeout`: Max wait time for ongoing calculations (default: `0`)
224

225
**Use Cases**:
226
- Enterprise applications with existing database infrastructure
227
- Audit trails and cache inspection requirements
228
- Integration with existing data management policies
229
- Long-term cache persistence with backup/recovery
230

231
**Advantages**:
232
- Leverages existing database infrastructure
233
- ACID properties and transactional consistency
234
- Advanced querying and management capabilities
235
- Integration with enterprise monitoring and backup
236

237
**Limitations**:
238
- Database overhead for simple caching needs
239
- Requires SQLAlchemy dependency
240
- Potential performance overhead compared to specialized cache stores
241

242
## Backend Selection Guide
243

244
### Choose Pickle When:
245
- Single-machine deployment
246
- Complex Python objects to cache
247
- Simple setup with no external dependencies
248
- Development and testing environments
249

250
### Choose Memory When:
251
- Highest performance requirements
252
- Temporary caching within application lifecycle
253
- Single-process applications
254
- Frequent cache access patterns
255

256
### Choose MongoDB When:
257
- Multi-machine distributed caching needed
258
- Document-oriented data fits well
259
- MongoDB already in infrastructure
260
- Complex query requirements on cached data
261

262
### Choose Redis When:
263
- High-performance distributed caching required
264
- Real-time applications
265
- Built-in expiration features needed
266
- Redis already in infrastructure
267

268
### Choose SQL When:
269
- Enterprise database infrastructure exists
270
- ACID properties required
271
- Advanced querying and reporting on cache needed
272
- Integration with existing data governance
273

274
## Usage Examples
275

276
### Backend Comparison
277

278
```python
279
from cachier import cachier
280
import time
281

282
# Same function with different backends
283
def expensive_computation(n):
284
    time.sleep(1)  # Simulate expensive operation
285
    return sum(i**2 for i in range(n))
286

287
# Pickle - for persistent single-machine caching
288
@cachier(backend='pickle')
289
def pickle_version(n):
290
    return expensive_computation(n)
291

292
# Memory - for fastest access
293
@cachier(backend='memory') 
294
def memory_version(n):
295
    return expensive_computation(n)
296

297
# MongoDB - for distributed caching
298
@cachier(backend='mongo', mongetter=get_mongo_collection)
299
def mongo_version(n):
300
    return expensive_computation(n)
301

302
# Redis - for high-performance distributed caching
303
@cachier(backend='redis', redis_client=get_redis_client)
304
def redis_version(n):
305
    return expensive_computation(n)
306

307
# SQL - for enterprise database-backed caching  
308
@cachier(backend='sql', sql_engine='sqlite:///cache.db')
309
def sql_version(n):
310
    return expensive_computation(n)
311
```
312

313
### Production Configuration Examples
314

315
```python
316
import os
317
from cachier import cachier, set_global_params
318

319
# Development environment - use memory for speed
320
if os.getenv('ENV') == 'development':
321
    set_global_params(backend='memory')
322

323
# Production environment - use Redis cluster
324
elif os.getenv('ENV') == 'production':
325
    import redis
326
    redis_client = redis.Redis(
327
        host=os.getenv('REDIS_HOST', 'cache-cluster'),
328
        port=int(os.getenv('REDIS_PORT', 6379)),
329
        password=os.getenv('REDIS_PASSWORD'),
330
        ssl=True
331
    )
332
    set_global_params(
333
        backend='redis',
334
        redis_client=redis_client,
335
        wait_for_calc_timeout=60
336
    )
337

338
# Testing environment - use pickle with temp directory
339
else:
340
    import tempfile
341
    set_global_params(
342
        backend='pickle',
343
        cache_dir=tempfile.mkdtemp()
344
    )
345

346
@cachier()  # Uses environment-appropriate backend
347
def application_function(data):
348
    return process_data(data)
349
```
350

351
### Multi-Backend Strategy
352

353
```python
354
from cachier import cachier
355

356
# Fast local cache for frequently accessed data
357
@cachier(backend='memory', stale_after=timedelta(minutes=5))
358
def local_fast_cache(key):
359
    # Check distributed cache first
360
    return distributed_cache(key)
361

362
# Distributed cache for shared data across instances
363
@cachier(backend='redis', redis_client=redis_client, stale_after=timedelta(hours=1))
364
def distributed_cache(key):
365
    # Fallback to persistent storage
366
    return persistent_cache(key)
367

368
# Persistent backup cache for rarely accessed data
369
@cachier(backend='sql', sql_engine=db_engine, stale_after=timedelta(days=1))
370
def persistent_cache(key):
371
    # Original data source
372
    return expensive_data_fetch(key)
373

374
# This creates a three-tier caching strategy:
375
# 1. Memory (fastest, 5min expiry)
376
# 2. Redis (fast distributed, 1hr expiry)  
377
# 3. SQL (persistent backup, 1day expiry)
378
# 4. Original source (slowest)
379
```
380

381
## Backend-Specific Type Definitions
382

383
```python { .api }
384
from typing import Callable, Union, TYPE_CHECKING
385

386
if TYPE_CHECKING:
387
    import pymongo.collection
388
    import redis
389

390
HashFunc = Callable[..., str]
391
Mongetter = Callable[[], "pymongo.collection.Collection"]
392
RedisClient = Union["redis.Redis", Callable[[], "redis.Redis"]]
393
Backend = Literal["pickle", "mongo", "memory", "redis", "sql"]
394
```
395

396
These type definitions ensure proper type checking and IDE support when configuring different backends.