CtrlK
BlogDocsLog inGet started
Tessl Logo

alonso-skills/postgres

Comprehensive PostgreSQL reference for developers and DBAs covering versions 14–18. Use whenever the user asks about PostgreSQL syntax, DDL/DML/DQL, joins, LATERAL, CTEs, window functions, GROUPING SETS, DISTINCT ON, RETURNING, ON CONFLICT, PL/pgSQL, functions, procedures, triggers, views, materialized views, indexes (B-tree/GIN/GiST/BRIN/Hash/Bloom), MVCC, VACUUM, autovacuum, WAL, TOAST, partitioning, replication (streaming/logical), backup, PITR, HA (Patroni/repmgr), pgBouncer, EXPLAIN ANALYZE, RLS, roles, extensions (pgvector, PostGIS, TimescaleDB, Citus, pg_trgm, pg_cron), JSON/JSONB, full-text search, UUID, timestamptz, COPY, system catalogs, collations, large objects, cursors, GUC, or any Postgres administration, performance, security, replication, backup, or recovery topic.

94

1.36x
Quality

94%

Does it follow best practices?

Impact

94%

1.36x

Average score across 3 eval scenarios

SecuritybySnyk

Passed

No known issues

Overview
Quality
Evals
Security
Files

80-connection-pooling.mdreferences/

Connection Pooling

[!WARNING] Pooling is mandatory at any non-trivial concurrency PostgreSQL forks one OS process per backend (~5-15 MB RSS each, more under load). Cluster running with max_connections = 100 and 100 active sessions costs at least 500 MB-1.5 GB RAM in backend processes alone — before work_mem, before shared_buffers, before parallel workers. Application that opens 500-2000 connections without pooler will exhaust memory long before saturating CPU. Pooling = solved problem with three established patterns: app-side, sidecar (pgBouncer), centralized.

Production reference for connection pooling: process-per-connection cost, sizing formulas, three pool modes, feature trade-offs, pgBouncer overview (deep dive lives in 81-pgbouncer.md).

Table of Contents

When to Use This Reference

Use this file for:

  • Sizing pool (how many connections to allow / accept)
  • Picking pool mode (session / transaction / statement)
  • Deciding pooling tier (app-side / sidecar / centralized)
  • Understanding what session-level features each mode breaks
  • Diagnosing "too many connections" + thundering-herd + lock-explosion problems

Use 81-pgbouncer.md for pgBouncer config + ops deep dive. Use 63-internals-architecture.md for process model + procarray cost. Use 46-roles-privileges.md for per-role pool routing patterns.

Mental Model

Five rules:

  1. Process-per-connection, not thread-per-connection. Postmaster forks one backend per client1. Backend = full OS process + ProcArray slot + per-backend memory (work_mem allocations per node, prepared-statement plan cache, GUC state, etc.). Fork latency + memory cost scale linearly with connections. Cannot raise max_connections past ~500-1000 on typical multi-core server hardware (spinning or SSD, non-HBM RAM) without operationally significant overhead. Pooler decouples app concurrency from backend count.

  2. Pool sizing formula = active connections, not max connections. Community wiki rule: active connections ≈ (CPU cores × 2) + effective_spindle_count2. Verbatim: "for optimal throughput the number of active connections should be somewhere near ((core_count * 2) + effective_spindle_count)". NVMe / fully-cached datasets → spindle term = 0. Pool size of 16-50 typically saturates 8-16-core machine. Bigger pool = lock contention + context switching + memory pressure, not more throughput.

  3. Three pool modes form a feature/throughput dial. Session mode (one client = one backend for connection lifetime, transparent, expensive). Transaction mode (backend lease per transaction, default for pgBouncer, loses session-level state). Statement mode (lease per statement, breaks multi-statement transactions, rare). Picking transaction over session = ~10-100× connection-reuse multiplier; cost = giving up SET/LISTEN/cursors-WITH-HOLD/advisory-session-locks.

  4. PG17 + libpq close-prepared = transaction-mode prepared statements work. Pre-PG17 + pgBouncer transaction mode: every prepared statement ties client to backend, defeats pooling. PG17 added PQclosePrepared() / PQclosePortal()3; pgBouncer 1.21+ (Oct 2023, supports protocol-level Close even on older servers) tracks named prepared statements across server-side pool members. Transaction-mode + prepared statements no longer mutually exclusive.

  5. Idle-in-transaction kills cluster; idle-outside-transaction merely annoys. idle_in_transaction_session_timeout (PG default 0 = disabled) kills sessions holding open transactions while idle — these block VACUUM, hold locks, retain xmin horizon. idle_session_timeout PG14+ kills idle-not-in-transaction sessions — only annoying (slot held, memory consumed, no horizon impact). Set the former aggressively (30s-5min); set the latter only with explicit pooler-awareness4.

Decision Matrix

Need / situationTier + modeAvoidWhy
App workload < 50 concurrent users, short transactionsApp-side pool + direct PG, session modeAdding pgBouncer adds complexity for nothingPool overhead > savings
Web app, > 100 concurrent users, short transactionsApp-side pool + pgBouncer transaction modeSession mode for 1000 clients to 50 backendsTransaction mode = mandatory for ratio >10:1
Need session-level state (SET, LISTEN, advisory locks, temp tables across transactions)pgBouncer session mode OR direct PGTransaction mode + SET — breaks silentlyTransaction mode disallows SET
Need protocol-level prepared statements + transaction-mode poolingPG17+ libpq + pgBouncer 1.21+ + max_prepared_statements > 0Pre-PG17 + transaction-mode + named PREPAREPre-PG17 ties client to backend
Batch jobs with one long transaction eachApp-side pool, low pool_size, session modeTransaction mode (no benefit for one-tx-per-job)Pooling adds nothing for long single tx
Serverless / Lambda / FaaS with bursty connection stormspgBouncer or pgcat in front; never raw libpqEach Lambda invocation opening + dropping connectionFork cost + procarray contention = catastrophic
Multi-tenant SaaS with per-tenant rolePool per role/tenant or transaction-mode pool with SET LOCAL onlyOne global pool with SET ROLE on checkoutWasted server-side memory; pool fragmentation
HA cluster with promoted standbypgBouncer in front of HAProxy → PG primaryApp direct to PG; needs reconnect on every failoverPooler = thin abstraction over failover
Microservices, > 50 services connecting to shared PGCentralized pgBouncer cluster; service connects to pgBouncerEach service opens own connection poolEach-service pool × N-services = explosion
Want session-state isolation per client but small backend countsession-mode pool, pool_size small (10-30)Transaction mode if any code uses SETSession mode preserves correctness
Pool checkout / return should reset session stateserver_reset_query = DISCARD ALL in pgBouncerNo reset query, transaction modeReset query catches stragglers
Replica scaling for readspgBouncer per-replica OR HAProxy in frontSingle pgBouncer across primary + replicas without per-pool routingpgBouncer not topology-aware

Three smell signals:

  1. pg_stat_activity shows hundreds of idle rows with no app workload → no pooler in front of cluster.
  2. Application throughput plateaus + p99 latency rises sharply when concurrent clients increase → pool too large (lock contention) or no pool (procarray contention).
  3. pg_stat_activity shows idle in transaction rows older than now() - interval '1 minute' → idle-in-transaction not bounded; VACUUM is being blocked. Set idle_in_transaction_session_timeout = '60s' per-role or cluster-wide.

Mechanics

Why pooling is needed

Postmaster forks one backend per accepted connection1:

"PostgreSQL implements a 'process per user' client/server model. In this model, every client process connects to exactly one backend process."

"we have to use a 'supervisor process' that spawns a new backend process every time a connection is requested. This supervisor process is called postmaster."

Per-backend cost:

Cost componentApproximate sizeNotes
Backend process RSS (idle)5-15 MBSteady-state; grows with workload
Backend process RSS (active)20-200 MB+work_mem allocations per node, plan cache, prepared statements
ProcArray slot1 entryLinear-scan cost for every snapshot creation; PG14 scalability fix mitigated5 but still scales with max_connections
Shared lock-table slotmax_locks_per_transactionCluster-wide, per-backend reserved
Fork latency1-5 msNew connection = postmaster fork + auth + startup packet

max_connections default is 1006:

"The default is typically 100 connections, but might be less if your kernel settings will not support it (as determined during initdb). This parameter can only be set at server start."

Restart-only. Cannot raise without bounce. Plan capacity ahead.

superuser_reserved_connections (default 3) — reserves slots for superusers when max_connections is full6.

Pool sizing formula

Wiki rule (canonical, oft-cited)2:

"for optimal throughput the number of active connections should be somewhere near ((core_count * 2) + effective_spindle_count)"

"Core count should not include HT threads, even if hyperthreading is enabled. Effective spindle count is zero if the active data set is fully cached, and approaches the actual number of spindles as the cache hit rate falls."

Tier table:

HardwareActive-connection targetNotes
4-core, NVMe, dataset cached8-10(4 × 2) + 0
8-core, NVMe, dataset cached16-20(8 × 2) + 0
16-core, NVMe, dataset cached32-40(16 × 2) + 0
8-core, mixed RAID-10 SAS, dataset 5× RAM18-24(8 × 2) + 6-8
32-core, NVMe, OLTP heavy64-80Diminishing returns past this; lock contention

Pool size ≠ max_connections. Common deployment: max_connections = 200, pgBouncer default_pool_size = 20, app sees 2000 concurrent clients. Backend count saturates around 20-40 active.

Wait queue size matters more than pool size at saturation. pgBouncer reserve_pool_size = 5, reserve_pool_timeout = 3s lets temporary spikes use spare slots. Beyond that → 429 / 503 / connection-refused at app layer.

Three pooling tiers

TierWhere it runsExamplesWhen to pick
App-sideIn-process inside applicationHikariCP (JVM), psycopg pool (Python), node-postgres pool (Node.js), pgx + Pgxpool (Go)Single app, < 50 connections, no inter-process sharing needed
SidecarOne pooler per app instancepgBouncer on app-server localhost, Unix socketPer-host pooling; bounds connection count per host
CentralizedDedicated pooler servicepgBouncer cluster behind LB, pgcatMany services sharing one PG, > 100 client processes total

Trade-off: app-side has zero network hop + zero auth latency + zero TLS termination cost, but limited to one process. Centralized adds 0.1-1 ms latency per query but consolidates connection budget across many app processes.

Combine tiers. Common: app-side pool of 10-20 connections per process → pgBouncer sidecar (transaction mode, pool_size 20) → PG primary. App-side enforces same-process connection reuse + retries; pgBouncer enforces backend-count cap.

Three pool modes

pgBouncer terminology (industry standard)7:

ModeLease unitVerbatim description
SessionClient connection lifetime"Most polite method. When a client connects, a server connection will be assigned to it for the whole duration the client stays connected."
TransactionSingle transaction"A server connection is assigned to a client only during a transaction. When PgBouncer notices that transaction is over, the server connection will be put back into the pool."
StatementSingle statement"Most aggressive method. The server connection will be put back into the pool immediately after a query completes. Multi-statement transactions are disallowed in this mode as they would break the assumption of statement pooling."

Default and almost-always-right choice = transaction mode. Lease per transaction = highest reuse + correct semantics for stateless web requests.

Session mode is mandatory when client code:

  • Issues SET / RESET outside transactions (per-session GUC)
  • Uses LISTEN / NOTIFY (cross-transaction)
  • Holds session-level advisory locks (pg_advisory_lock, not pg_advisory_xact_lock)
  • Uses session-lifetime temporary tables (ON COMMIT PRESERVE ROWS / DELETE ROWS)
  • Issues SQL-level PREPARE / DEALLOCATE (not protocol-level prepared statements)
  • Uses WITH HOLD cursors

Statement mode = rare. Only useful for true single-statement workloads (read-only analytics with autocommit = true, no transactions). Most ORMs implicitly start transactions; statement mode breaks them.

Pool mode × feature compatibility matrix

Verbatim from pgBouncer features matrix8:

FeatureSessionTransaction
Startup parameters (client_encoding, datestyle, timezone, standard_conforming_strings, application_name)YesYes
SET / RESETYesNever
LISTENYesNever
NOTIFYYesYes
WITHOUT HOLD CURSORYesYes
WITH HOLD CURSORYesNever
Protocol-level prepared plansYesYes (pgBouncer 1.21+)
SQL-level PREPARE / DEALLOCATEYesNever
ON COMMIT DROP temp tablesYesYes
PRESERVE/DELETE ROWS temp tablesYesNever
Cached plan resetYesYes
LOAD statementYesNever
Session-level advisory locksYesNever

Transaction-mode rule of thumb: anything that survives COMMIT and persists at the session level is broken. Use SET LOCAL instead of SET. Use pg_advisory_xact_lock instead of pg_advisory_lock. Use ON COMMIT DROP temp tables only.

Statement-mode is transaction-mode minus multi-statement transactions. pgBouncer features.html does not publish a separate statement column — statement mode inherits transaction restrictions plus the usage.html rule: "Multi-statement transactions are disallowed in this mode as they would break the assumption of statement pooling."

reserved_connections (PG16+)

[!NOTE] PostgreSQL 16 reserved_connections GUC added — reserves connection slots for roles that have the pg_use_reserved_connections predefined role granted9. Verbatim: "Allow the server to reserve backend slots for roles with pg_use_reserved_connections membership (Nathan Bossart). The number of reserved slots is set by server variable reserved_connections."

Three-tier reservation:

Reservation tierGUCReserved forDefault
Topsuperuser_reserved_connectionsSuperusers only3
Middle (PG16+)reserved_connectionsRoles with pg_use_reserved_connections0
BottomRemainingAll other rolesmax_connections - sum(above)

Use case: dedicated monitoring + replication + maintenance roles never get locked out when application pool exhausts max_connections. Grant pg_use_reserved_connections to those roles, set reserved_connections = 5.

Idle-session timeouts

GUCDefaultKills whatWhy it matters
idle_in_transaction_session_timeout0 (disabled)Sessions idle inside transactionCritical. Blocks VACUUM, holds xmin horizon, retains locks. Set to 30s-5min cluster-wide or per-role
idle_session_timeout (PG14+)0 (disabled)Sessions idle, no transaction openCosmetic / capacity. Pooler-aware before setting cluster-wide4
statement_timeout0 (disabled)Single statement running too longPer-query; per-role baseline pattern
lock_timeout0 (disabled)Waiting for lock too longPer-query; per-role baseline pattern

[!WARNING] idle_session_timeout + pooler Verbatim docs warning: "Be wary of enforcing this timeout on connections made through connection-pooling software or other middleware, as such a layer may not react well to unexpected connection closure." pgBouncer + cluster-wide idle_session_timeout = pgBouncer reconnects silently but logs grow. Set only per-role for non-pooled roles, or align with pgBouncer server_idle_timeout.

[!NOTE] PostgreSQL 14 idle_session_timeout introduced10. Verbatim: "Add server parameter idle_session_timeout to close idle sessions (Li Japin). This is similar to idle_in_transaction_session_timeout."

Pooler landscape

PoolerLanguagePool modesMaturityNotes
pgBouncerCsession / transaction / statementBattle-tested since 2007De-facto standard. Single-threaded but very efficient. See 81-pgbouncer.md
Pgpool-IICsession / connection-pool / replication / load-balanceMature; broader feature setHeavier; also does query routing, load balancing, in-memory cache. Heavier ops surface
pgcatRustsession / transactionActive; postgresml/pgcatMulti-threaded; supports sharding + per-shard routing; newer than pgBouncer
OdysseyCsession / transactionYandex-developedMulti-threaded; less mainstream than pgBouncer
awslabs/pgbouncer-fast-switchoverC (pgBouncer fork)session / transactionAWS-maintained"intercept and programmatically change client queries before they are sent to the server" — adds query rewriting + failover speedup
App-side poolslanguage-nativesession-equivalent onlyLibrary-dependentHikariCP (Java), psycopg pool (Python), node-postgres (Node), pgx (Go)

Default recommendation = pgBouncer. Smallest operational surface, broadest production evidence, transaction mode + PG17 close-prepared = mature. Pgpool-II + pgcat are valid for specific use cases (load balancing, sharding) but heavier.

Per-Version Timeline

VersionConnection-pooling-relevant itemsSources
PG14idle_session_timeout GUC introduced (Li Japin)10; snapshot scalability improvements reduce per-backend cost at high max_connections5PG14 release notes
PG15No direct pooling-relevant changesPG15 release notes
PG16reserved_connections GUC + pg_use_reserved_connections predefined role (Nathan Bossart)9; pg_hba.conf include directives + regex-on-database/user (composes with pooler-injected application_name for per-app rules)PG16 release notes
PG17PQclosePrepared() / PQclosePortal() libpq functions (Jelte Fennema-Nio)3 — explicit support for pgBouncer-style closing of named prepared statements + portals; pgBouncer 1.21+ (Oct 2023) tracks named prepared statements across pool members via protocol-level Close (not strictly requiring PG17, but PG17 libpq helpers make it canonical for app drivers); pg_dump/pgBouncer + scram-passthrough improvements; event_triggers GUC disables event triggers cluster-wide for debugging (cross-reference 40-event-triggers.md)PG17 release notes, pgBouncer changelog
PG18No headline connection-pooling-feature changes; async I/O subsystem reduces per-backend I/O wait11PG18 release notes

[!NOTE] PG17 + pgBouncer 1.21 = transaction-mode prepared statements Pre-PG17, transaction-mode + prepared statements = mutually exclusive (each PREPARE ties client to backend). pgBouncer 1.21 (Oct 2023, pre-PG17 GA) added protocol-level named-prepared-statement tracking. PG17 added libpq PQclosePrepared() so client drivers can portably close prepared statements. Set max_prepared_statements = 100-200 in pgBouncer config12:

"Add support for protocol-level named prepared statements! This is probably one of the most requested features for PgBouncer ... In synthetic benchmarks this feature was able to increase query throughput anywhere from 15% to 250%, depending on the workload."

Examples / Recipes

Recipe 1 — Baseline production setup: 8-core OLTP cluster

# postgresql.conf
max_connections = 200                  # Bound. App reaches via pgBouncer, not direct
superuser_reserved_connections = 3
reserved_connections = 5               # PG16+: for monitoring/replication roles

idle_in_transaction_session_timeout = '60s'  # Critical
statement_timeout = '0'                 # Set per-role
lock_timeout = '0'                      # Set per-role
# pgbouncer.ini
[databases]
appdb = host=127.0.0.1 port=5432 dbname=appdb

[pgbouncer]
pool_mode = transaction
listen_port = 6432
listen_addr = 127.0.0.1
auth_type = scram-sha-256
auth_file = /etc/pgbouncer/userlist.txt

default_pool_size = 20                  # ~ (cores × 2) + 4 buffer
max_client_conn = 2000                  # 100× backend count
reserve_pool_size = 5
reserve_pool_timeout = 3
server_reset_query = DISCARD ALL        # Clean state between client leases
server_idle_timeout = 600                # Close backend idle for 10 min

max_prepared_statements = 200            # PG17+ libpq + pgBouncer 1.21+

Three-config-block convention: postgresql.conf + pgBouncer config + app driver connection string. App connects to 127.0.0.1:6432 not :5432.

Recipe 2 — Pool sizing calculation

-- Determine target pool size per host
WITH params AS (
    SELECT
        (SELECT setting::int FROM pg_settings WHERE name = 'max_connections') AS max_conn,
        4 AS cpu_cores,                  -- adjust for actual cores
        0 AS effective_spindles           -- 0 = fully cached / NVMe
)
SELECT
    max_conn,
    cpu_cores,
    effective_spindles,
    (cpu_cores * 2 + effective_spindles) AS recommended_active_pool,
    max_conn / (cpu_cores * 2 + effective_spindles) AS recommended_clients_per_backend
FROM params;

Output guides default_pool_size in pgBouncer + max_clients_per_pool heuristics.

Recipe 3 — Per-role pool routing (sidecar pgBouncer)

# pgbouncer.ini — separate pools per role for different workload profiles
[databases]
appdb_web = host=127.0.0.1 port=5432 dbname=appdb pool_size=30 pool_mode=transaction
appdb_reporter = host=127.0.0.1 port=5432 dbname=appdb pool_size=5 pool_mode=session
appdb_batch = host=127.0.0.1 port=5432 dbname=appdb pool_size=3 pool_mode=session

Web tier connects to appdb_web (transaction mode, large pool). Reporter role connects to appdb_reporter (session mode for long analytic queries with WITH HOLD cursors). Batch jobs connect to appdb_batch (small pool, session mode for one-tx-per-job).

Recipe 4 — Detect "no pooler" symptom

-- High idle-connection count = no pooler
SELECT
    state,
    count(*) AS sessions,
    pg_size_pretty(count(*) * 10 * 1024 * 1024) AS approx_rss
FROM pg_stat_activity
WHERE backend_type = 'client backend'
GROUP BY state
ORDER BY sessions DESC;

idle rows greater than ~50 with sub-1% CPU utilization → app keeps connections open between requests. Pool tier missing or misconfigured.

Recipe 5 — Detect idle-in-transaction (the canonical bloat builder)

SELECT
    pid,
    usename,
    application_name,
    client_addr,
    now() - xact_start AS xact_duration,
    now() - state_change AS idle_duration,
    state,
    query
FROM pg_stat_activity
WHERE state = 'idle in transaction'
  AND xact_start IS NOT NULL
ORDER BY xact_duration DESC;

Any xact_duration > 1 minute = bug. Set idle_in_transaction_session_timeout = '60s' cluster-wide or per-role. Cross-reference 27-mvcc-internals.md for xmin-horizon consequences.

Recipe 6 — Connection-storm protection at app boundary

# Python (psycopg pool) — bound at app side, sidecar pgBouncer further bounds
import psycopg_pool

pool = psycopg_pool.ConnectionPool(
    conninfo="host=127.0.0.1 port=6432 dbname=appdb",   # → pgBouncer
    min_size=2,
    max_size=10,
    timeout=5.0,                # Refuse-fast under load
    max_lifetime=900,           # Recycle every 15 min
    max_idle=300,               # Close idle after 5 min
    reconnect_timeout=30,
)

Two-tier bound: app pool max 10 → pgBouncer default_pool_size 20 → PG max_connections 200. App-side timeout=5.0 returns 503 to user fast instead of saturating backend.

Recipe 7 — Per-role timeouts (canonical baseline)

-- Webapp role: aggressive timeouts (correctness)
ALTER ROLE webapp SET statement_timeout = '5s';
ALTER ROLE webapp SET lock_timeout = '500ms';
ALTER ROLE webapp SET idle_in_transaction_session_timeout = '30s';

-- Reporter role: long queries allowed, no idle-in-tx tolerated
ALTER ROLE reporter SET statement_timeout = '15min';
ALTER ROLE reporter SET idle_in_transaction_session_timeout = '30s';
ALTER ROLE reporter SET default_transaction_read_only = on;

-- Batch role: long-running, monitoring picks up failures
ALTER ROLE batchjobs SET statement_timeout = '2h';
ALTER ROLE batchjobs SET idle_in_transaction_session_timeout = '5min';

[!NOTE] Per-role + pgBouncer transaction mode caveat Per-role ALTER ROLE SET values do NOT propagate across pgBouncer transaction-mode pool returns. pgBouncer's server_reset_query = DISCARD ALL may clear them. Cross-reference 46-roles-privileges.md gotcha #6. Mitigations: (a) make the value cluster-wide in postgresql.conf, or (b) set in pgBouncer's connect_query, or (c) use SET LOCAL per transaction.

Recipe 8 — Migrate from session to transaction mode (verification checklist)

-- Pre-migration audit: find session-level state usage
-- 1. SQL-level prepared statements (PREPARE / EXECUTE / DEALLOCATE)
SELECT query FROM pg_stat_statements
WHERE query ILIKE 'PREPARE %' OR query ILIKE 'DEALLOCATE %';

-- 2. Session-level advisory locks (vs pg_advisory_xact_lock)
SELECT query FROM pg_stat_statements
WHERE query ~* 'pg_advisory_lock\s*\('
  AND query !~* 'pg_advisory_xact_lock';

-- 3. SET vs SET LOCAL
SELECT query FROM pg_stat_statements
WHERE query ~* '^SET '
  AND query !~* '^SET LOCAL';

-- 4. LISTEN
SELECT query FROM pg_stat_statements
WHERE query ILIKE 'LISTEN %';

-- 5. WITH HOLD cursors
SELECT query FROM pg_stat_statements
WHERE query ILIKE '%WITH HOLD%';

-- 6. Session-lifetime temp tables
SELECT query FROM pg_stat_statements
WHERE query ~* 'CREATE\s+TEMP'
  AND query !~* 'ON COMMIT DROP';

Each non-empty result = code change needed before flipping pool_mode = transaction.

Recipe 9 — Verify pgBouncer transaction-mode prepared statements (PG17+)

-- After enabling max_prepared_statements > 0 in pgBouncer 1.21+
-- and connecting via app driver that uses protocol-level prepared statements:

-- pgBouncer console (psql -p 6432 pgbouncer)
SHOW STATS;
SHOW POOLS;

-- Server-side verification via pg_stat_statements
-- Same query repeated should show calls > 1 with stable queryid (queryid stable across sessions/clients).

App drivers using protocol-level extended-query messages (most modern drivers) benefit automatically. SQL-level PREPARE still requires session mode.

Recipe 10 — Three-tier pooling at scale (microservices + central pool)

[Service A pod] [Service B pod] [Service C pod]   ← N replicas each, app-pool size 5
       ↓                ↓                ↓
       └────────────────┼────────────────┘
                        ↓
                [pgBouncer cluster (3-5 replicas)]   ← transaction mode, pool_size 30
                        ↓
                [HAProxy / Patroni REST]              ← failover
                        ↓
                [PG primary + 2 standbys]             ← max_connections 200

App pool × N services × M replicas = client side. pgBouncer cluster bounds total backend usage. HAProxy routes around failover.

Recipe 11 — Monitor pool exhaustion (server side)

-- Backend usage per role
SELECT
    rolname,
    count(*) AS backends,
    count(*) FILTER (WHERE state = 'active') AS active,
    count(*) FILTER (WHERE state = 'idle in transaction') AS idle_in_tx
FROM pg_stat_activity a
JOIN pg_roles r ON r.oid = a.usesysid
GROUP BY rolname
ORDER BY backends DESC;

-- Current vs max
SELECT
    count(*) AS current_backends,
    (SELECT setting::int FROM pg_settings WHERE name = 'max_connections') AS max_connections,
    round(100.0 * count(*) /
        (SELECT setting::int FROM pg_settings WHERE name = 'max_connections'), 1) AS pct_used
FROM pg_stat_activity
WHERE backend_type = 'client backend';

Alert at 75% pct_used. Hit 100% = "FATAL: sorry, too many clients already" errors.

Recipe 12 — Monitor pool exhaustion (pgBouncer side)

$ psql -h 127.0.0.1 -p 6432 -U pgbouncer pgbouncer
pgbouncer=# SHOW POOLS;
 database | user    | cl_active | cl_waiting | sv_active | sv_idle | sv_used | maxwait
----------+---------+-----------+------------+-----------+---------+---------+---------
 appdb    | webapp  | 1500      | 50         | 18        | 2       | 0       | 12

pgbouncer=# SHOW STATS;

cl_waiting > 0 = clients queued; pool is saturated. maxwait > 5 (seconds) = users seeing slow responses. Raise default_pool_size or reserve_pool_size, or scale pgBouncer horizontally.

Recipe 13 — Connection-pool checkout reset patterns

# pgBouncer — clean state on server return to pool
server_reset_query = DISCARD ALL

DISCARD ALL resets: temp tables, prepared statements, cursors, listen channels, session GUCs (back to defaults), advisory session locks. Safe + canonical. Equivalent to fresh connection state.

Alternative (PG14+, narrower):

DISCARD PLANS;            -- only prepared-plan cache
DISCARD TEMP;             -- only temp tables
DISCARD SEQUENCES;        -- only sequence cache

Use full DISCARD ALL unless you have a specific reason. Faster than reconnecting.

Gotchas / Anti-patterns

  1. Setting max_connections = 1000 and skipping pooler. Linear procarray scan, 5-15 GB+ RSS, lock-table pressure. Pool. Always pool.

  2. Pool size equal to max_connections. Defeats pooling — every client gets own backend. Pool size should be 10-100× smaller than max_connections.

  3. Transaction-mode pooling + SET (not SET LOCAL). Silently breaks: GUC clears on next transaction's pool return. Code that uses SET search_path outside transactions in transaction-mode pool = bug.

  4. pg_advisory_lock in transaction-mode pool. Lock held in backend, but client's next transaction may land on different backend. Lock orphan + correctness bug. Use pg_advisory_xact_lock.

  5. SQL-level PREPARE / EXECUTE in transaction-mode pool. Prepared name lives only on that backend. Next transaction may not find it. Use protocol-level prepared statements (PG17+ libpq + pgBouncer 1.21+).

  6. idle_in_transaction_session_timeout = 0 (default). Means: any client bug that holds open transaction → VACUUM blocked indefinitely, xmin horizon held back, bloat accumulates. Set to 30s-5min cluster-wide. Setting this GUC is a high-impact, low-risk change for most production deployments; it prevents indefinite transaction holds that silently accumulate bloat.

  7. idle_session_timeout cluster-wide + pgBouncer. Verbatim docs warning. pgBouncer reconnects silently but logs grow + reconnect overhead. Set per-role for non-pooled access only, or align with server_idle_timeout.

  8. Pool too small → app threads block waiting for connection. Symptom: app-side connection timeouts; pgBouncer cl_waiting > 0; queries themselves fast. Raise pool size or reduce app concurrency.

  9. Pool too large → server-side lock contention + memory pressure. Symptom: pg_stat_activity shows many active rows; CPU pegged; per-query latency rising. Reduce pool size.

  10. One global pool, many roles, role-switching via SET ROLE on checkout. Wastes per-backend role-cache; planner cache pollution. Run separate pgBouncer pools per role.

  11. pgBouncer + cluster-wide event_triggers GUC (PG17+) set off. Some apps depend on event triggers for audit/cache invalidation. Verify before disabling. Cross-reference 40-event-triggers.md.

  12. No server_reset_query. Connection state leaks between client leases — prepared statements, GUCs, advisory locks. Set server_reset_query = DISCARD ALL.

  13. Application connecting directly to PG when pgBouncer is also deployed. Bypasses pool entirely. Either remove direct path, or accept dual-connect for specific session-mode workloads.

  14. max_prepared_statements = 0 (pgBouncer default) + protocol-level prepared statements. pgBouncer 1.21+ supports them in transaction mode only when this GUC is non-zero. Default zero = feature disabled, every PREPARE re-sent.

  15. max_connections raise without reboot plan. Postmaster-context GUC. Requires restart, not reload. Plan capacity ahead.

  16. Per-role ALTER ROLE SET GUCs not propagating through transaction-mode pool. Set in postgresql.conf, in pgBouncer connect_query, or use SET LOCAL per transaction. Cross-reference 46-roles-privileges.md gotcha #6.

  17. Pool size scaled with concurrent users instead of CPU cores. Wrong formula. CPU cores × 2 + spindles, not user count. Concurrent users connects to pool entry queue, not backend.

  18. Sidecar pgBouncer with listen_addr = * and no firewall. pgBouncer authenticates clients, but accepting external connections without TLS exposes credentials. Bind to 127.0.0.1 or use TLS.

  19. Statement-mode pooling with ORM that implicitly opens transactions. Hibernate, ActiveRecord, SQLAlchemy default to transactions per request. Statement mode breaks them. Use transaction mode.

  20. Multi-tenant SaaS using transaction-mode pool + role switching mid-transaction. Role active when transaction commits is what gets logged + applied for pg_stat_activity.usename. Use one pool per tenant role.

  21. Stale connection in pool after PG restart. pgBouncer detects + reconnects, but in-flight transactions error. Application must retry on connection-reset errors. Set query_wait_timeout for clean failure.

  22. HAProxy + pgBouncer + Patroni stack without health-check awareness. pgBouncer doesn't know if it's pointing at primary or standby. HAProxy must route based on Patroni REST API check (/primary or /replica), not on pgBouncer alone. Cross-reference 79-patroni.md Recipe 6.

  23. Replicas pooled identically to primary. Replicas often serve read-only workload with lower concurrency. Smaller per-replica pool (5-10) is fine. Don't blindly mirror primary pool config.

See Also

Sources

Additional sources consulted:

Footnotes

  1. PostgreSQL 16 documentation, Chapter 52.2 "How Connections Are Established". Verbatim: "PostgreSQL implements a 'process per user' client/server model. In this model, every client process connects to exactly one backend process." and "we have to use a 'supervisor process' that spawns a new backend process every time a connection is requested. This supervisor process is called postmaster." https://www.postgresql.org/docs/16/connect-estab.html 2

  2. PostgreSQL Wiki, "Number Of Database Connections". Verbatim: "for optimal throughput the number of active connections should be somewhere near ((core_count * 2) + effective_spindle_count)" and "Core count should not include HT threads, even if hyperthreading is enabled. Effective spindle count is zero if the active data set is fully cached, and approaches the actual number of spindles as the cache hit rate falls." https://wiki.postgresql.org/wiki/Number_Of_Database_Connections 2

  3. PostgreSQL 17 release notes, section E.10.3.7. Verbatim: "Add libpq functions to close portals and prepared statements (Jelte Fennema-Nio). The functions are PQclosePrepared(), PQclosePortal(), PQsendClosePrepared(), and PQsendClosePortal()." https://www.postgresql.org/docs/release/17.0/ 2

  4. PostgreSQL 16 documentation, idle_session_timeout GUC description. Verbatim: "Be wary of enforcing this timeout on connections made through connection-pooling software or other middleware, as such a layer may not react well to unexpected connection closure." https://www.postgresql.org/docs/16/runtime-config-client.html 2

  5. PostgreSQL 14 release notes. Snapshot scalability improvements by Andres Freund — reduce ProcArray contention at high max_connections. https://www.postgresql.org/docs/release/14.0/ 2

  6. PostgreSQL 16 documentation, Section 20.3 "Connections and Authentication". Verbatim: "The default is typically 100 connections, but might be less if your kernel settings will not support it (as determined during initdb). This parameter can only be set at server start." https://www.postgresql.org/docs/16/runtime-config-connection.html 2

  7. pgBouncer usage documentation, pool-mode descriptions. https://www.pgbouncer.org/usage.html

  8. pgBouncer feature compatibility matrix. https://www.pgbouncer.org/features.html

  9. PostgreSQL 16 release notes, section E.14.3.1.5. Verbatim: "Allow the server to reserve backend slots for roles with pg_use_reserved_connections membership (Nathan Bossart). The number of reserved slots is set by server variable reserved_connections." https://www.postgresql.org/docs/release/16.0/ 2

  10. PostgreSQL 14 release notes, section E.23.3.1.9. Verbatim: "Add server parameter idle_session_timeout to close idle sessions (Li Japin). This is similar to idle_in_transaction_session_timeout." https://www.postgresql.org/docs/release/14.0/ 2

  11. PostgreSQL 18 release announcement. Async I/O subsystem reduces per-backend I/O wait via io_method + worker pool. https://www.postgresql.org/about/news/postgresql-18-released-3142/

  12. pgBouncer changelog, 1.21.0 release (2023-10-16). Verbatim: "Add support for protocol-level named prepared statements! This is probably one of the most requested features for PgBouncer." and "In synthetic benchmarks this feature was able to increase query throughput anywhere from 15% to 250%, depending on the workload." and "To benefit from this new feature you need to change the new max_prepared_statements setting to a non-zero value (the exact value depends on your workload, but 100 is probably reasonable)." https://www.pgbouncer.org/changelog.html

references

01-syntax-ddl.md

02-syntax-dql.md

03-syntax-dml.md

04-ctes.md

05-views.md

06-functions.md

07-procedures.md

08-plpgsql.md

09-procedural-languages.md

10-dynamic-sql.md

11-window-functions.md

12-aggregates-grouping.md

13-cursors-and-prepares.md

14-data-types-builtin.md

15-data-types-custom.md

16-arrays.md

17-json-jsonb.md

18-uuid-numeric-money.md

19-timestamp-timezones.md

20-text-search.md

21-hstore.md

22-indexes-overview.md

23-btree-indexes.md

24-gin-gist-indexes.md

25-brin-hash-spgist-bloom-indexes.md

26-index-maintenance.md

27-mvcc-internals.md

28-vacuum-autovacuum.md

29-transaction-id-wraparound.md

30-hot-updates.md

31-toast.md

32-buffer-manager.md

33-wal.md

34-checkpoints-bgwriter.md

35-partitioning.md

36-inheritance.md

37-constraints.md

38-foreign-keys-deep.md

39-triggers.md

40-event-triggers.md

41-transactions.md

42-isolation-levels.md

43-locking.md

44-advisory-locks.md

45-listen-notify.md

46-roles-privileges.md

47-row-level-security.md

48-authentication-pg-hba.md

49-tls-ssl.md

50-encryption-pgcrypto.md

51-pgaudit.md

52-rules-system.md

53-server-configuration.md

54-memory-tuning.md

55-statistics-planner.md

56-explain.md

57-pg-stat-statements.md

58-performance-diagnostics.md

59-planner-tuning.md

60-parallel-query.md

61-jit-compilation.md

62-tablespaces.md

63-internals-architecture.md

64-system-catalogs.md

65-collations-encoding.md

66-bulk-operations-copy.md

67-cli-tools.md

68-pgbench.md

69-extensions.md

70-fdw.md

71-large-objects.md

72-extension-development.md

73-streaming-replication.md

74-logical-replication.md

75-replication-slots.md

76-logical-decoding.md

77-standby-failover.md

78-ha-architectures.md

79-patroni.md

80-connection-pooling.md

81-pgbouncer.md

82-monitoring.md

83-backup-pg-dump.md

84-backup-physical-pitr.md

85-backup-tools.md

86-pg-upgrade.md

87-major-version-upgrade.md

88-corruption-recovery.md

89-pg-rewind.md

90-disaster-recovery.md

91-docker-postgres.md

92-kubernetes-operators.md

93-pg-trgm.md

94-pgvector.md

95-postgis.md

96-timescaledb.md

97-citus.md

98-pg-cron.md

99-pg-partman.md

100-pg-versions-features.md

101-managed-vs-baremetal.md

102-skill-cookbook.md

SKILL.md

tile.json