SQLite best practices for Python -- PRAGMAs per connection, context manager transactions, parameterized queries, row_factory, executemany, FK indexes
92
90%
Does it follow best practices?
Impact
97%
1.73xAverage score across 5 eval scenarios
Passed
No known issues
{
"context": "Tests whether the agent proactively applies SQLite best practices when building a Python inventory tracker. The task describes business requirements (atomic shipments, stock validation) but never mentions PRAGMAs, WAL mode, foreign keys, context managers, row_factory, parameterized queries, or indexes. The agent should apply all of these from knowledge of SQLite+Python gotchas.",
"type": "weighted_checklist",
"checklist": [
{
"name": "WAL mode enabled",
"description": "Connection setup sets PRAGMA journal_mode=WAL",
"max_score": 10
},
{
"name": "Foreign keys ON",
"description": "Connection setup sets PRAGMA foreign_keys=ON (must be set per connection, not once globally)",
"max_score": 10
},
{
"name": "Busy timeout set",
"description": "Connection setup sets PRAGMA busy_timeout to a positive value (e.g. 5000)",
"max_score": 6
},
{
"name": "Row factory configured",
"description": "Connection sets conn.row_factory = sqlite3.Row for dict-like access to query results",
"max_score": 8
},
{
"name": "Receive shipment uses context manager transaction",
"description": "Receiving a shipment (multi-item stock update + movement records) uses 'with conn:' context manager for atomic commit/rollback -- not manual conn.commit()",
"max_score": 12
},
{
"name": "Record sale uses context manager with validation",
"description": "Recording a sale checks stock levels and updates quantities within a 'with conn:' context manager, rolling back if any item has insufficient stock",
"max_score": 12
},
{
"name": "Parameterized queries throughout",
"description": "ALL queries use ? placeholders -- no f-strings, .format(), or string concatenation for SQL values",
"max_score": 10
},
{
"name": "executemany for bulk inserts",
"description": "Bulk operations (inserting multiple movement records) use conn.executemany() rather than looping with individual execute() calls",
"max_score": 6
},
{
"name": "Indexes on FK columns",
"description": "Explicit CREATE INDEX on stock_movements foreign key column referencing products",
"max_score": 8
},
{
"name": "Money as INTEGER cents",
"description": "Unit cost and sell price stored as INTEGER (cents), not REAL or TEXT",
"max_score": 6
},
{
"name": "Connection properly closed",
"description": "Connections are closed in finally blocks, or connection lifecycle is managed via dependency injection / context manager -- no leaked connections",
"max_score": 6
},
{
"name": "Dates as ISO TEXT strings",
"description": "Timestamps stored as TEXT in ISO 8601 format, date range queries use string comparison or datetime() functions",
"max_score": 6
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
skills
sqlite-python-best-practices
verifiers