0
# Cache Policy and Expiration
1
2
Cache policy and expiration features provide flexible control over when responses are cached, how long they remain valid, and how cache validation works. This includes support for HTTP Cache-Control headers, URL-specific patterns, conditional requests, and custom expiration logic.
3
4
## Capabilities
5
6
### Cache Settings
7
8
Central configuration class that controls all aspects of cache behavior.
9
10
```python { .api }
11
class CacheSettings:
12
"""Internal settings class for cache behavior configuration."""
13
14
def __init__(
15
self,
16
allowable_codes: Iterable[int] = (200,),
17
allowable_methods: Iterable[str] = ('GET', 'HEAD'),
18
always_revalidate: bool = False,
19
cache_control: bool = False,
20
disabled: bool = False,
21
expire_after: ExpirationTime = -1,
22
filter_fn: Optional[FilterCallback] = None,
23
ignored_parameters: Optional[Iterable[str]] = None,
24
key_fn: Optional[KeyCallback] = None,
25
match_headers: Union[Iterable[str], bool] = False,
26
stale_if_error: Union[bool, int] = False,
27
stale_while_revalidate: Union[bool, int] = False,
28
urls_expire_after: Optional[ExpirationPatterns] = None,
29
**kwargs
30
):
31
"""
32
Configure cache behavior settings.
33
34
Parameters:
35
- allowable_codes: Only cache responses with these status codes
36
- allowable_methods: Only cache these HTTP methods
37
- always_revalidate: Always validate cached responses with server
38
- cache_control: Use HTTP Cache-Control headers for expiration
39
- disabled: Temporarily disable all caching
40
- expire_after: Default expiration time for all cached responses
41
- filter_fn: Custom function to determine what responses to cache
42
- ignored_parameters: Parameters to exclude from cache keys
43
- key_fn: Custom function for generating cache keys
44
- match_headers: Headers to include in cache keys for matching
45
- stale_if_error: Return stale responses when new requests fail
46
- stale_while_revalidate: Return stale responses while refreshing in background
47
- urls_expire_after: URL-specific expiration patterns
48
"""
49
50
@classmethod
51
def from_kwargs(cls, **kwargs) -> 'CacheSettings':
52
"""Create settings instance from keyword arguments."""
53
```
54
55
### Expiration Time Functions
56
57
Functions for converting various expiration time formats into standardized values.
58
59
```python { .api }
60
def get_expiration_datetime(expire_after: ExpirationTime) -> Optional[datetime]:
61
"""
62
Convert expiration value to absolute datetime.
63
64
Parameters:
65
- expire_after: Expiration time in various formats
66
67
Returns:
68
Absolute expiration datetime or None for no expiration
69
70
Accepts:
71
- int/float: seconds from now
72
- str: ISO datetime or relative time ('1 hour', '30 minutes')
73
- datetime: absolute expiration time
74
- timedelta: relative time from now
75
- None: no expiration
76
- -1: never expire (NEVER_EXPIRE)
77
- 0: expire immediately (EXPIRE_IMMEDIATELY)
78
"""
79
80
def get_expiration_seconds(expire_after: ExpirationTime) -> Optional[float]:
81
"""
82
Convert expiration value to seconds from now.
83
84
Parameters:
85
- expire_after: Expiration time in various formats
86
87
Returns:
88
Seconds until expiration or None for no expiration
89
"""
90
91
def get_url_expiration(
92
url: str,
93
urls_expire_after: ExpirationPatterns
94
) -> ExpirationTime:
95
"""
96
Get URL-specific expiration time from pattern matching.
97
98
Parameters:
99
- url: Request URL to match
100
- urls_expire_after: Dict mapping URL patterns to expiration times
101
102
Returns:
103
Expiration time for matching pattern or None if no match
104
105
Pattern matching supports:
106
- Glob patterns: '*.example.com/api/*'
107
- Regex patterns: compiled regex objects
108
- Exact URLs: 'https://api.example.com/data'
109
"""
110
111
def add_tzinfo(dt: datetime, timezone: Optional[tzinfo] = None) -> datetime:
112
"""Add timezone info to naive datetime (defaults to UTC)."""
113
114
def utcnow() -> datetime:
115
"""Get current UTC time with timezone info."""
116
```
117
118
#### Usage Examples
119
120
Basic expiration configuration:
121
122
```python
123
from requests_cache import CachedSession
124
from datetime import datetime, timedelta
125
126
# Simple numeric expiration (seconds)
127
session = CachedSession('cache', expire_after=3600) # 1 hour
128
129
# Using timedelta objects
130
session = CachedSession('cache', expire_after=timedelta(hours=2))
131
132
# Using datetime objects (absolute expiration)
133
expire_time = datetime.now() + timedelta(days=1)
134
session = CachedSession('cache', expire_after=expire_time)
135
136
# Using string formats
137
session = CachedSession('cache', expire_after='1 hour')
138
session = CachedSession('cache', expire_after='30 minutes')
139
session = CachedSession('cache', expire_after='2023-12-31T23:59:59')
140
```
141
142
URL-specific expiration patterns:
143
144
```python
145
from requests_cache import CachedSession
146
147
session = CachedSession(
148
'cache',
149
expire_after=3600, # Default: 1 hour
150
urls_expire_after={
151
# Fast-changing APIs: 5 minutes
152
'*.fastapi.com/data': 300,
153
'https://api.realtime.com/*': '5 minutes',
154
155
# Slow-changing data: 1 day
156
'*.static.com/*': timedelta(days=1),
157
158
# Different expiration for different endpoints
159
'https://api.example.com/user/*': timedelta(hours=1),
160
'https://api.example.com/posts/*': timedelta(minutes=30),
161
162
# Never expire certain responses
163
'https://api.example.com/constants': -1,
164
165
# Using regex patterns
166
re.compile(r'.*\.example\.com/v\d+/data'): '1 hour'
167
}
168
)
169
```
170
171
### Cache Actions
172
173
Class that translates cache settings and HTTP headers into specific cache actions for each request.
174
175
```python { .api }
176
class CacheActions:
177
"""Translates settings and headers into cache actions for requests."""
178
179
@classmethod
180
def from_request(
181
cls,
182
cache_key: str,
183
request: AnyPreparedRequest,
184
settings: CacheSettings
185
) -> 'CacheActions':
186
"""Create cache actions based on request and settings."""
187
188
def update_from_cached_response(
189
self,
190
cached_response: Optional[CachedResponse],
191
create_key_fn: Callable,
192
**kwargs
193
) -> None:
194
"""Update actions based on cached response state."""
195
196
def update_from_response(self, response: AnyResponse) -> None:
197
"""Update actions based on new response headers."""
198
199
@property
200
def cache_key(self) -> str:
201
"""Cache key for this request."""
202
203
@property
204
def error_504(self) -> bool:
205
"""Return 504 error instead of making request."""
206
207
@property
208
def expire_after(self) -> ExpirationTime:
209
"""Computed expiration time for this request."""
210
211
@property
212
def send_request(self) -> bool:
213
"""Send new HTTP request."""
214
215
@property
216
def resend_request(self) -> bool:
217
"""Resend request to refresh stale cached response."""
218
219
@property
220
def resend_async(self) -> bool:
221
"""Resend request asynchronously while using stale response."""
222
223
@property
224
def skip_read(self) -> bool:
225
"""Skip reading from cache."""
226
227
@property
228
def skip_write(self) -> bool:
229
"""Skip writing response to cache."""
230
```
231
232
### HTTP Cache Directives
233
234
Parser for HTTP Cache-Control and related headers that affect caching behavior.
235
236
```python { .api }
237
class CacheDirectives:
238
"""Parses and stores HTTP cache control directives."""
239
240
def __init__(self, headers: Mapping[str, str]):
241
"""
242
Parse cache directives from HTTP headers.
243
244
Parameters:
245
- headers: HTTP response headers
246
247
Parsed directives include:
248
- Cache-Control header values
249
- Expires header
250
- ETag header
251
- Last-Modified header
252
"""
253
254
# Cache-Control directive properties
255
@property
256
def expires(self) -> Optional[datetime]:
257
"""Expiration time from Expires header."""
258
259
@property
260
def immutable(self) -> bool:
261
"""immutable directive."""
262
263
@property
264
def max_age(self) -> Optional[int]:
265
"""max-age directive value in seconds."""
266
267
@property
268
def max_stale(self) -> Optional[int]:
269
"""max-stale directive value in seconds."""
270
271
@property
272
def min_fresh(self) -> Optional[int]:
273
"""min-fresh directive value in seconds."""
274
275
@property
276
def must_revalidate(self) -> bool:
277
"""must-revalidate directive."""
278
279
@property
280
def no_cache(self) -> bool:
281
"""no-cache directive."""
282
283
@property
284
def no_store(self) -> bool:
285
"""no-store directive."""
286
287
@property
288
def only_if_cached(self) -> bool:
289
"""only-if-cached directive."""
290
291
@property
292
def stale_if_error(self) -> Optional[int]:
293
"""stale-if-error directive value in seconds."""
294
295
@property
296
def stale_while_revalidate(self) -> Optional[int]:
297
"""stale-while-revalidate directive value in seconds."""
298
299
# Validation headers
300
@property
301
def etag(self) -> Optional[str]:
302
"""ETag header value."""
303
304
@property
305
def last_modified(self) -> Optional[str]:
306
"""Last-Modified header value."""
307
308
def set_request_headers(
309
headers: Optional[MutableMapping[str, str]],
310
expire_after: ExpirationTime = None,
311
only_if_cached: bool = False,
312
refresh: bool = False,
313
force_refresh: bool = False
314
) -> MutableMapping[str, str]:
315
"""
316
Convert request parameters to appropriate HTTP headers.
317
318
Parameters:
319
- headers: Existing request headers
320
- expire_after: Override expiration for this request
321
- only_if_cached: Add Cache-Control: only-if-cached
322
- refresh: Add Cache-Control: max-age=0
323
- force_refresh: Add Cache-Control: no-cache
324
325
Returns:
326
Updated headers dict
327
"""
328
```
329
330
#### Usage Examples
331
332
HTTP cache control integration:
333
334
```python
335
from requests_cache import CachedSession
336
337
# Enable HTTP cache control header processing
338
session = CachedSession(
339
'cache',
340
cache_control=True, # Respect Cache-Control headers
341
expire_after=3600 # Fallback expiration
342
)
343
344
# Server response with Cache-Control: max-age=1800
345
# Will be cached for 1800 seconds regardless of expire_after setting
346
response = session.get('https://api.example.com/data')
347
348
# Server response with Cache-Control: no-cache
349
# Will not be cached regardless of settings
350
response = session.get('https://api.example.com/nocache')
351
```
352
353
Per-request cache control:
354
355
```python
356
# Force refresh (ignore cached version)
357
response = session.get(
358
'https://api.example.com/data',
359
force_refresh=True
360
)
361
362
# Soft refresh (revalidate with server)
363
response = session.get(
364
'https://api.example.com/data',
365
refresh=True
366
)
367
368
# Only return if cached (return 504 if not cached)
369
response = session.get(
370
'https://api.example.com/data',
371
only_if_cached=True
372
)
373
374
# Override expiration for this request
375
response = session.get(
376
'https://api.example.com/data',
377
expire_after=300 # 5 minutes
378
)
379
```
380
381
### Constants and Defaults
382
383
Predefined constants for common expiration values and default settings.
384
385
```python { .api }
386
# Expiration constants
387
DO_NOT_CACHE: int # Special value to disable caching for specific responses
388
EXPIRE_IMMEDIATELY: int = 0 # Expire immediately
389
NEVER_EXPIRE: int = -1 # Never expire
390
391
# Default settings
392
DEFAULT_CACHE_NAME: str = 'http_cache'
393
DEFAULT_METHODS: Tuple[str, ...] = ('GET', 'HEAD')
394
DEFAULT_STATUS_CODES: Tuple[int, ...] = (200,)
395
DEFAULT_IGNORED_PARAMS: Tuple[str, ...] = (
396
'Authorization',
397
'X-API-KEY',
398
'access_token',
399
'api_key'
400
)
401
```
402
403
#### Usage Examples
404
405
Using expiration constants:
406
407
```python
408
from requests_cache import CachedSession, NEVER_EXPIRE, EXPIRE_IMMEDIATELY
409
410
session = CachedSession(
411
'cache',
412
expire_after=NEVER_EXPIRE, # Never expire by default
413
urls_expire_after={
414
'*.temp.com/*': EXPIRE_IMMEDIATELY, # Always fetch fresh
415
'*.static.com/*': NEVER_EXPIRE, # Never expire
416
}
417
)
418
```
419
420
Custom filtering and key generation:
421
422
```python
423
from requests_cache import CachedSession
424
425
def should_cache_response(response):
426
"""Only cache successful responses from trusted domains."""
427
if response.status_code != 200:
428
return False
429
if 'trusted.com' not in response.url:
430
return False
431
return True
432
433
def custom_cache_key(*args, **kwargs):
434
"""Generate cache key that ignores user-specific parameters."""
435
# Custom key generation logic
436
return f"custom_key_{hash(args)}"
437
438
session = CachedSession(
439
'cache',
440
filter_fn=should_cache_response,
441
key_fn=custom_cache_key,
442
ignored_parameters=['user_id', 'session_token']
443
)
444
```
445
446
## Types
447
448
```python { .api }
449
# Expiration types
450
ExpirationTime = Union[None, int, float, str, datetime, timedelta]
451
ExpirationPattern = Union[str, Pattern] # Glob string or compiled regex
452
ExpirationPatterns = Dict[ExpirationPattern, ExpirationTime]
453
454
# Callback types
455
FilterCallback = Callable[[Response], bool]
456
KeyCallback = Callable[..., str]
457
458
# Header type
459
HeaderDict = MutableMapping[str, str]
460
```