Persistent, stale-free, local and cross-machine caching for Python functions.
—
Quality
Pending
Does it follow best practices?
Impact
Pending
No eval scenarios have been run
The cachier decorator is the main interface for adding persistent caching to Python functions. It provides comprehensive memoization with multiple backend support, configurable expiration, and advanced features like stale-aware caching and background refresh.
Simple persistent caching with default pickle backend:
def cachier(
hash_func: Optional[HashFunc] = None,
hash_params: Optional[HashFunc] = None, # Deprecated
backend: Optional[Backend] = None,
mongetter: Optional[Mongetter] = None,
sql_engine: Optional[Union[str, Any, Callable[[], Any]]] = None,
redis_client: Optional[RedisClient] = None,
stale_after: Optional[timedelta] = None,
next_time: Optional[bool] = None,
cache_dir: Optional[Union[str, os.PathLike]] = None,
pickle_reload: Optional[bool] = None,
separate_files: Optional[bool] = None,
wait_for_calc_timeout: Optional[int] = None,
allow_none: Optional[bool] = None,
cleanup_stale: Optional[bool] = None,
cleanup_interval: Optional[timedelta] = None,
):
"""
Wrap as a persistent, stale-free memoization decorator.
The positional and keyword arguments to the wrapped function must be
hashable (i.e. Python's immutable built-in objects, not mutable
containers).
Parameters:
- hash_func: A callable that gets the args and kwargs from the decorated
function and returns a hash key for them. Enables use with
functions that get arguments not automatically hashable.
- hash_params: Deprecated, use hash_func instead
- backend: Backend name ('pickle', 'mongo', 'memory', 'sql', 'redis').
Defaults to 'pickle' unless a core-associated parameter is provided.
- mongetter: A callable that takes no arguments and returns a
pymongo.Collection object with writing permissions.
If unset, local pickle cache is used.
- sql_engine: SQLAlchemy connection string, Engine, or callable returning
an Engine. Used for the SQL backend.
- redis_client: Redis client instance or callable returning a Redis client.
Used for the Redis backend.
- stale_after: Time delta after which a cached result is considered stale.
Calls made after result goes stale trigger recalculation.
- next_time: If True, stale result returned when finding one, not waiting
for fresh result calculation. Defaults to False.
- cache_dir: Fully qualified path to cache file directory. Process must
have running permissions. Defaults to XDG-compliant cache directory
or ~/.cachier/ as fallback
- pickle_reload: If True, in-memory cache reloaded on each read, enabling
different threads to share cache. Should be False for
faster reads in single-thread programs. Defaults to True.
- separate_files: For Pickle cores only. Instead of single cache file per
function, each function's cache split between several files,
one for each argument set. Helps with large cache files.
- wait_for_calc_timeout: For MongoDB only. Maximum time to wait for ongoing
calculation. When process starts calculation setting
being_calculated to True, any process trying to read
same entry waits maximum of seconds specified.
0 means wait forever.
- allow_none: Allows storing None values in cache. If False, functions
returning None not cached and recalculated every call.
- cleanup_stale: If True, stale cache entries periodically deleted in
background thread. Defaults to False.
- cleanup_interval: Minimum time between automatic cleanup runs.
Defaults to one day.
Returns:
Decorated function with attached cache management methods:
- clear_cache(): Clear all cached entries
- clear_being_calculated(): Mark all entries as not being calculated
- cache_dpath(): Return cache directory path if exists
- precache_value(*args, value_to_cache, **kwargs): Add initial value to cache
"""Usage examples:
from cachier import cachier
from datetime import timedelta
import os
# Basic caching with default pickle backend
@cachier()
def compute_heavy_task(data):
# Expensive computation
return sum(x**2 for x in data)
# Custom cache directory
@cachier(cache_dir="/tmp/my_cache")
def process_files(file_path):
with open(file_path, 'r') as f:
return f.read().upper()
# Stale-aware caching with background refresh
@cachier(
stale_after=timedelta(minutes=30),
next_time=True # Return stale while refreshing in background
)
def fetch_external_data(api_endpoint):
import requests
return requests.get(api_endpoint).json()
# Custom hash function for complex objects
def custom_hash(args, kwargs):
# Custom hashing logic for non-hashable arguments
import pickle
import hashlib
serialized = pickle.dumps((args, sorted(kwargs.items())))
return hashlib.sha256(serialized).hexdigest()
@cachier(hash_func=custom_hash)
def analyze_dataframe(df, options):
# Process pandas DataFrame (not naturally hashable)
return df.groupby('category').sum()
# Cleanup stale entries automatically
@cachier(
stale_after=timedelta(hours=1),
cleanup_stale=True,
cleanup_interval=timedelta(minutes=30)
)
def periodic_report(date_range):
# Generate reports with automatic cleanup
return generate_report(date_range)Functions can control caching behavior at runtime using special keyword arguments:
# Skip cache for specific call
result = my_cached_function(data, cachier__skip_cache=True)
# Overwrite cached value
result = my_cached_function(data, cachier__overwrite_cache=True)
# Enable verbose cache logging
result = my_cached_function(data, cachier__verbose=True)
# Override stale_after for specific call
result = my_cached_function(data, cachier__stale_after=timedelta(minutes=5))
# Override next_time behavior
result = my_cached_function(data, cachier__next_time=False)
# Set maximum age for this specific call
result = my_cached_function(data, max_age=timedelta(minutes=10))Runtime Parameters:
cachier__skip_cache: Skip cache lookup and storage for this callcachier__overwrite_cache: Force recalculation and overwrite existing cache entrycachier__verbose: Enable verbose logging for this callcachier__stale_after: Override decorator's stale_after setting for this callcachier__next_time: Override decorator's next_time setting for this callcachier__cleanup_stale: Override decorator's cleanup_stale setting for this callcachier__cleanup_interval: Override decorator's cleanup_interval setting for this callcachier__allow_none: Override decorator's allow_none setting for this callmax_age: Maximum allowed age for cached value (triggers recalculation if exceeded)All decorated functions receive these attached methods:
def clear_cache() -> None:
"""Clear all cached entries for this function."""def clear_being_calculated() -> None:
"""Mark all entries in this cache as not being calculated."""def cache_dpath() -> Optional[str]:
"""Return the path to the cache directory, if exists; None if not."""def precache_value(*args, value_to_cache, **kwargs):
"""
Add an initial value to the cache.
Parameters:
- *args, **kwargs: Function arguments to associate with cached value
- value_to_cache: Entry to be written into the cache
Returns:
The cached value
"""Usage examples:
@cachier()
def expensive_function(x, y):
return x * y + complex_calculation(x, y)
# Pre-populate cache with known values
expensive_function.precache_value(5, 10, value_to_cache=150)
# Clear specific function's cache
expensive_function.clear_cache()
# Get cache location
cache_path = expensive_function.cache_dpath()
print(f"Cache stored at: {cache_path}")
# Reset calculation locks (useful after crashes)
expensive_function.clear_being_calculated()Install with Tessl CLI
npx tessl i tessl/pypi-cachier