Our CPU was at 12%. Our database was idle. Every request still hung for 30 seconds. We weren't out of resources. We were out of connections.

Problem

Response times spiked under normal traffic. Every dashboard we checked (CPU, memory, database load) looked fine. Requests still queued for 30+ seconds before timing out.

Why it happens

• The connection pool was sized for local dev traffic, not production concurrency, so it capped out well below what the database could actually handle
• A slow endpoint held its connection for the full length of the request instead of releasing it early
• One error path was missing a finally block, so failed requests leaked a connection that never returned to the pool
• Nothing monitored pool utilization directly, so the exhaustion was invisible until requests started queuing

Better approach

• Monitor pool checkout time and active-vs-idle connections as first-class metrics, not just CPU and DB load
• Size the pool based on load testing at expected peak concurrency, not a default copied from a tutorial
• Wrap every connection checkout in a try/finally (or a context manager) so an exception can’t leak it
• Set a checkout timeout so a starved pool fails fast and loud instead of queuing silently

Example

A pool capped at 10 connections held up fine at 8 concurrent requests. At 11, the next request waited for a connection to free up, even though the database itself could have handled 200 concurrent queries without breaking a sweat.