tessl/pypi-jinja2

A very fast and expressive template engine for Python applications

—

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Securityby

Pending

The risk profile of this skill

Overview

Eval results

Files

Bytecode Caching

Name: tessl/pypi-jinja2
Author: tessl

Performance optimization system that caches compiled template bytecode to filesystem or external storage systems like Memcached. Significantly improves template loading performance in production environments.

Capabilities

Bytecode Cache Base Class

Abstract base class for implementing bytecode caching strategies with pluggable storage backends.

class BytecodeCache:
    def load_bytecode(self, bucket):
        """
        Load bytecode from cache into bucket.
        
        Parameters:
            bucket: Bucket instance to load bytecode into
        """
    
    def dump_bytecode(self, bucket):
        """
        Save bytecode from bucket to cache.
        
        Parameters:
            bucket: Bucket instance containing bytecode to save
        """
    
    def clear(self):
        """
        Clear entire cache.
        """
    
    def get_cache_key(self, name, filename=None):
        """
        Generate cache key for template.
        
        Parameters:
            name: Template name
            filename: Template filename (optional)
            
        Returns:
            str: Cache key string
        """
    
    def get_source_checksum(self, source):
        """
        Generate checksum for template source.
        
        Parameters:
            source: Template source code
            
        Returns:
            str: Source checksum
        """
    
    def get_bucket(self, environment, name, filename, source):
        """
        Get cache bucket for template.
        
        Parameters:
            environment: Jinja2 environment instance
            name: Template name
            filename: Template filename
            source: Template source code
            
        Returns:
            Bucket: Cache bucket instance
        """
    
    def set_bucket(self, bucket):
        """
        Store cache bucket.
        
        Parameters:
            bucket: Bucket instance to store
        """

Filesystem Bytecode Cache

Filesystem-based bytecode caching for persistent local storage of compiled templates.

class FileSystemBytecodeCache(BytecodeCache):
    def __init__(self, directory=None, pattern='__jinja2_%s.cache'):
        """
        Initialize filesystem bytecode cache.
        
        Parameters:
            directory: Cache directory path (default: system temp directory)
            pattern: Filename pattern with %s placeholder for cache key
        """

Usage example:

from jinja2 import Environment, FileSystemLoader, FileSystemBytecodeCache
import tempfile
import os

# Create cache directory
cache_dir = os.path.join(tempfile.gettempdir(), 'jinja2_cache')
os.makedirs(cache_dir, exist_ok=True)

# Setup environment with bytecode cache
env = Environment(
    loader=FileSystemLoader('templates'),
    bytecode_cache=FileSystemBytecodeCache(cache_dir)
)

# First load compiles and caches
template1 = env.get_template('page.html')  # Slow: compile + cache
template2 = env.get_template('page.html')  # Fast: load from cache

Memcached Bytecode Cache

Memcached-based bytecode caching for distributed caching across multiple application instances.

class MemcachedBytecodeCache(BytecodeCache):
    def __init__(self, client, prefix='jinja2/bytecode/', timeout=None, ignore_memcache_errors=True):
        """
        Initialize Memcached bytecode cache.
        
        Parameters:
            client: Memcached client instance
            prefix: Key prefix for cache entries (default: 'jinja2/bytecode/')
            timeout: Cache timeout in seconds (default: None for no expiration)
            ignore_memcache_errors: Ignore Memcached errors (default: True)
        """

Usage example:

from jinja2 import Environment, FileSystemLoader, MemcachedBytecodeCache
import memcache

# Setup Memcached client
mc = memcache.Client(['127.0.0.1:11211'])

# Setup environment with Memcached cache
env = Environment(
    loader=FileSystemLoader('templates'),
    bytecode_cache=MemcachedBytecodeCache(mc, timeout=3600)  # 1 hour timeout
)

template = env.get_template('page.html')

Cache Bucket

Cache storage unit that manages bytecode serialization and metadata.

class Bucket:
    def __init__(self, environment, key, checksum):
        """
        Initialize cache bucket.
        
        Parameters:
            environment: Jinja2 environment instance
            key: Cache key
            checksum: Source checksum
        """
    
    def load_bytecode(self, f):
        """
        Load bytecode from file-like object.
        
        Parameters:
            f: File-like object to read from
        """
    
    def write_bytecode(self, f):
        """
        Write bytecode to file-like object.
        
        Parameters:
            f: File-like object to write to
        """
    
    def bytecode_from_string(self, string):
        """
        Load bytecode from byte string.
        
        Parameters:
            string: Bytecode as bytes
        """
    
    def bytecode_to_string(self):
        """
        Convert bytecode to byte string.
        
        Returns:
            bytes: Bytecode as bytes
        """
    
    def reset(self):
        """
        Clear bytecode from bucket.
        """

Custom Cache Implementation

Redis Bytecode Cache Example

Example implementation of a custom Redis-based bytecode cache:

import pickle
from jinja2.bccache import BytecodeCache, Bucket

class RedisBytecodeCache(BytecodeCache):
    def __init__(self, redis_client, prefix='jinja2:', timeout=3600):
        self.redis = redis_client
        self.prefix = prefix
        self.timeout = timeout
    
    def load_bytecode(self, bucket):
        key = self.prefix + bucket.key
        data = self.redis.get(key)
        if data is not None:
            bucket.bytecode_from_string(data)
    
    def dump_bytecode(self, bucket):
        key = self.prefix + bucket.key
        data = bucket.bytecode_to_string()
        if data is not None:
            self.redis.setex(key, self.timeout, data)
    
    def clear(self):
        keys = self.redis.keys(self.prefix + '*')
        if keys:
            self.redis.delete(*keys)

# Usage
import redis
r = redis.Redis()
env = Environment(
    loader=FileSystemLoader('templates'),
    bytecode_cache=RedisBytecodeCache(r)
)

Database Bytecode Cache Example

Example implementation using a database for bytecode storage:

import sqlite3
from jinja2.bccache import BytecodeCache

class SQLiteBytecodeCache(BytecodeCache):
    def __init__(self, database_path):
        self.db_path = database_path
        self._init_database()
    
    def _init_database(self):
        conn = sqlite3.connect(self.db_path)
        conn.execute('''
            CREATE TABLE IF NOT EXISTS bytecode_cache (
                key TEXT PRIMARY KEY,
                bytecode BLOB,
                checksum TEXT,
                created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
            )
        ''')
        conn.commit()
        conn.close()
    
    def load_bytecode(self, bucket):
        conn = sqlite3.connect(self.db_path)
        cursor = conn.execute(
            'SELECT bytecode FROM bytecode_cache WHERE key = ? AND checksum = ?',
            (bucket.key, bucket.checksum)
        )
        row = cursor.fetchone()
        conn.close()
        
        if row:
            bucket.bytecode_from_string(row[0])
    
    def dump_bytecode(self, bucket):
        data = bucket.bytecode_to_string()
        if data is not None:
            conn = sqlite3.connect(self.db_path)
            conn.execute(
                'INSERT OR REPLACE INTO bytecode_cache (key, bytecode, checksum) VALUES (?, ?, ?)',
                (bucket.key, data, bucket.checksum)
            )
            conn.commit()
            conn.close()
    
    def clear(self):
        conn = sqlite3.connect(self.db_path)
        conn.execute('DELETE FROM bytecode_cache')
        conn.commit()
        conn.close()

Performance Considerations

Cache Effectiveness

Bytecode caching provides the most benefit when:

Templates are reused frequently
Template compilation is a bottleneck
Templates contain complex logic or many includes/extends
Running in production with stable templates

Cache Invalidation

Caches automatically invalidate when:

Template source files change (filesystem modification time)
Template source content changes (checksum comparison)
Environment configuration changes
Jinja2 version changes

Memory vs. Disk Trade-offs

# In-memory cache for development
env = Environment(
    loader=FileSystemLoader('templates'),
    cache_size=100,  # Keep 100 compiled templates in memory
    auto_reload=True  # Auto-reload on changes
)

# Disk cache for production
env = Environment(
    loader=FileSystemLoader('templates'),
    bytecode_cache=FileSystemBytecodeCache('/var/cache/jinja2'),
    cache_size=0,     # Disable memory cache
    auto_reload=False # Disable auto-reload
)

Types

class CacheStatistics:
    """
    Cache performance statistics for monitoring and optimization.
    
    Attributes:
        hits: Number of cache hits
        misses: Number of cache misses
        hit_ratio: Cache hit ratio (hits / (hits + misses))
        size: Current cache size
        max_size: Maximum cache size
    """

class CacheEntry:
    """
    Individual cache entry with metadata.
    
    Attributes:
        key: Cache key
        checksum: Source checksum
        bytecode: Compiled bytecode
        created_at: Creation timestamp
        accessed_at: Last access timestamp
    """