SQLAlchemy patterns — engine setup, session management, declarative models,
98
99%
Does it follow best practices?
Impact
98%
1.96xAverage score across 5 eval scenarios
Passed
No known issues
{
"context": "Tests whether the agent proactively applies SQLAlchemy best practices when building a user/orders service. The task says nothing about pool configuration, eager loading, cascade rules, expire_on_commit, or Mapped types -- the agent should apply these patterns on its own.",
"type": "weighted_checklist",
"checklist": [
{
"name": "pool_pre_ping enabled",
"description": "Engine is created with pool_pre_ping=True to detect stale connections. The agent was NOT asked about connection pooling.",
"max_score": 10
},
{
"name": "Pool size and overflow configured",
"description": "Engine specifies pool_size, max_overflow, and pool_recycle rather than relying on defaults. The agent was NOT asked to tune pool settings.",
"max_score": 10
},
{
"name": "Database URL from environment variable",
"description": "Database URL is read from os.getenv or os.environ, not hardcoded as a string literal.",
"max_score": 8
},
{
"name": "expire_on_commit=False on sessionmaker",
"description": "sessionmaker (or async_sessionmaker) is configured with expire_on_commit=False to prevent DetachedInstanceError. The agent was NOT told about this pitfall.",
"max_score": 12
},
{
"name": "Session uses yield with finally for cleanup",
"description": "The session dependency function uses yield and closes the session in a finally block or uses a context manager for proper cleanup on both success and error.",
"max_score": 10
},
{
"name": "DeclarativeBase used (not declarative_base)",
"description": "Base class inherits from DeclarativeBase (SQLAlchemy 2.0 style), not the legacy declarative_base() function.",
"max_score": 8
},
{
"name": "Mapped types used (not Column)",
"description": "Model columns use Mapped[type] = mapped_column(...) syntax, not legacy Column() definitions. The task did not specify which SQLAlchemy version to use.",
"max_score": 10
},
{
"name": "Eager loading on get_user_with_orders",
"description": "The get_user_with_orders function uses selectinload, joinedload, or subqueryload to load orders in the same query, not relying on lazy loading. The agent was NOT asked about N+1 prevention.",
"max_score": 14
},
{
"name": "cascade='all, delete-orphan' on User.orders",
"description": "The User.orders relationship specifies cascade='all, delete-orphan' so orphaned orders are cleaned up. The agent was NOT told about cascade rules.",
"max_score": 10
},
{
"name": "ondelete='CASCADE' on ForeignKey",
"description": "Order.user_id ForeignKey includes ondelete='CASCADE' for database-level cascade deletion.",
"max_score": 8
},
{
"name": "back_populates used (not backref)",
"description": "The User-Order relationship uses back_populates for explicit bidirectional linking, not the implicit backref.",
"max_score": 6
},
{
"name": "Foreign key columns indexed",
"description": "Order.user_id has index=True for query performance.",
"max_score": 4
}
]
}