Connection Pooling in Distributed Systems
Connection pooling is a concept well understood in the monolith world: create the pool at startup, borrow a connection when a request arrives, return it to the pool when done, set its maximum limit, and the system runs stable. The problem arises when the same understanding is carried into a distributed system without questioning it. There, almost all the assumptions underlying the classic model collapse — and the collapse isn’t gradual, but sudden, in production, when traffic rises. This article discusses why the classic model fundamentally fails in distributed systems, how connection exhaustion happens on Lambda, and how RDS Proxy — and other alternatives — shift the pooling responsibility to the right layer.
Assumptions That Collapse in Distributed Systems
The classic connection pool works because of one fundamental assumption that’s almost never stated explicitly: the number of application instances accessing the database is a small, predictable number.
In a monolith with 3 instances behind a load balancer, you can calculate exactly how many maximum connections will open to the database: 3 instances × 20 connections per pool = 60 connections. This number won’t change unless you manually add instances. This makes configuring max_connections in PostgreSQL or MySQL easy and deterministic.
In distributed systems, especially serverless ones, all these numbers become variables that can fluctuate within seconds.
flowchart TD
subgraph Monolith["Monolith — Predictable"]
APP1[Instance 1\nPool: 20 conn] --> DB1[(Database\nmax_conn: 100)]
APP2[Instance 2\nPool: 20 conn] --> DB1
APP3[Instance 3\nPool: 20 conn] --> DB1
NOTE1["Total connections: 60\nDeterministic, safe"]
end
subgraph Serverless["Lambda — Unpredictable"]
L1[Lambda #1\nPool: 5 conn] --> DB2[(Database\nmax_conn: 100)]
L2[Lambda #2\nPool: 5 conn] --> DB2
L3[Lambda #3\nPool: 5 conn] --> DB2
LDOT["... up to\n500 instances"] --> DB2
NOTE2["Total connections: could be 2,500\nConnection exhaustion!"]
end
style NOTE2 fill:#ffebee,stroke:#e53935
style NOTE1 fill:#e8f5e9,stroke:#43a047This isn’t an edge case scenario. This is Lambda’s normal behavior when facing a traffic spike — and this is why connection pooling at the application level isn’t enough in serverless architectures.
The Anatomy of Connection Exhaustion on Lambda
To understand the problem precisely, we first need to understand how Lambda manages instances and database connections.
The Lambda Instance Lifecycle
Lambda doesn’t run all invocations in one process. Every concurrent execution requires its own instance, and each instance has a fully isolated runtime — including memory, global variables, and database connections.
stateDiagram-v2
[*] --> ColdStart: First invocation\nor capacity exhausted
ColdStart --> WarmInit: Container created\nRuntime initialized
WarmInit --> Running: Handler executed\nDB connection opened
Running --> Idle: Handler finished\nConnection stays open in the pool
Idle --> Running: Next invocation\n(warm start, connection reused)
Idle --> Terminated: Idle too long\nor Lambda reclaims
note right of Running: Every concurrent execution\nrequires a different instance
note right of Idle: Pool connections stay alive\nwhile the instance is idleThe crucial part of this diagram: database connections are opened at cold start and stay open while the instance is alive — even when no handler is running. This is desirable behavior in a monolith (connection reuse = efficiency), but on Lambda it creates a problem: hundreds of idle instances each holding a connection to the database.
The Connection Exhaustion Timeline
Here’s a scenario very commonly seen in production:
sequenceDiagram
participant Traffic as Traffic (spike)
participant Lambda as Lambda Instances
participant DB as PostgreSQL (max 100 conn)
Note over DB: max_connections = 100
Traffic->>Lambda: 10 concurrent requests
Lambda->>DB: 10 instances × 3 conn = 30 connections
Note over DB: 30/100 connections used — safe
Traffic->>Lambda: Traffic rises 5x
Lambda->>DB: 50 instances × 3 conn = 150 connections
Note over DB: 100/100 connections — database full!
Lambda->>DB: Instance #51 tries to connect
DB-->>Lambda: FATAL: sorry, too many clients already
Lambda-->>Traffic: HTTP 500 — Lambda timeout
Note over Lambda,DB: New instances keep being created\nBut can't connect\nAll requests failWhat makes the situation worse: Lambdas that fail to connect usually retry immediately, which just adds more pressure to an already overwhelmed database. This is a positive feedback loop toward total failure.
Why Code-Level Pools Don’t Help
The first reaction of many engineers facing this problem is reducing MaxOpenConns at the code level. This does reduce connections per instance — but doesn’t solve the fundamental problem.
// ANTI-PATTERN: thinking this is enough for Lambda
sqlDB.SetMaxOpenConns(5) // 5 connections per instance
// With 200 concurrent Lambdas: 200 × 5 = 1000 connections to the DB
// The database is still exhausted
// Half solution: reduce drastically
sqlDB.SetMaxOpenConns(1) // 1 connection per instance
// With 200 concurrent Lambdas: still 200 connections to the DB
// Better, but still uncontrolled and can't be capped
No MaxOpenConns value at the Lambda level can provide a definite upper bound for database connections — because you can’t control how many Lambda instances AWS creates.
Reducing MaxOpenConns on Lambda is a correct but insufficient optimization. It only reduces the severity of the problem, not eliminates it. As long as Lambda can scale to hundreds of instances, no pool setting at the code level can provide real protection against connection exhaustion.RDS Proxy: Moving Pooling to the Infrastructure
The right solution isn’t fixing pooling at the application level, but moving the pooling to a layer between the application and the database — a layer that can see and control all connections centrally, regardless of how many application instances exist.
RDS Proxy is AWS’s official implementation of this pattern.
How RDS Proxy Works
sequenceDiagram
participant L1 as Lambda Instance 1
participant L2 as Lambda Instance 2
participant L3 as Lambda Instance N
participant PROXY as RDS Proxy\n(Shared Pool)
participant RDS as RDS Database
L1->>PROXY: Connect (lightweight connection)
L2->>PROXY: Connect (lightweight connection)
L3->>PROXY: Connect (lightweight connection)
Note over PROXY: The proxy has a centralized,\nlimited connection pool to RDS
PROXY->>RDS: Only opens connections\nas needed\n(not per Lambda instance)
L1->>PROXY: Query: SELECT * FROM orders
PROXY->>RDS: Forward via an existing pooled connection
RDS-->>PROXY: Result
PROXY-->>L1: Result
L1->>PROXY: Query done, connection returned to the pool
Note over PROXY: Connection returned to the pool\nready for another Lambda
L2->>PROXY: Next query uses the same connectionThe fundamental difference: without a proxy, every Lambda instance maintains its own connection to the database. With a proxy, all Lambda instances connect to the proxy (very lightweight connections), and the proxy manages connections to the database centrally.
What RDS Proxy Provides
Besides centralized pooling, RDS Proxy provides several other benefits relevant to Lambda architectures:
Connection multiplexing. A single physical connection in the proxy pool can be used in turn by many Lambda instances that aren’t simultaneously querying. This is multiplexing that’s impossible at the application level.
Faster failover. When an RDS failover happens (primary down, replica promoted), the proxy handles reconnection transparently. Lambda doesn’t need to know a failover occurred.
IAM and Secrets Manager integration. RDS Proxy supports authentication using IAM tokens, so Lambda doesn’t need to store database credentials — credentials are managed by Secrets Manager and rotated automatically.
Pinning control. RDS Proxy performs connection pinning by default when it detects operations that can’t be multiplexed (like SET statements or stored procedures that change session state). Understanding when pinning happens is important for maximizing proxy efficiency.
RDS Proxy Setup: Step by Step
Infrastructure Prerequisites
Before creating an RDS Proxy, make sure the following components already exist:
Prerequisites:
✓ A running RDS instance (PostgreSQL / MySQL)
✓ An AWS Secrets Manager secret storing the database credentials
✓ The same VPC for Lambda, RDS Proxy, and RDS
✓ Correctly configured security groups
✓ An IAM role for RDS Proxy (Secrets Manager access)
Storing Credentials in Secrets Manager
RDS Proxy takes database credentials from Secrets Manager. This isn’t just for security — it’s an architectural requirement so the proxy can manage authentication centrally.
# Create a secret for the database credentials
aws secretsmanager create-secret \
--name "prod/myapp/rds-credentials" \
--description "RDS credentials for myapp production" \
--secret-string '{
"username": "app_user",
"password": "strong-password-here",
"engine": "postgres",
"host": "mydb.cluster-xyz.ap-southeast-1.rds.amazonaws.com",
"port": 5432,
"dbname": "app_db"
}' \
--region ap-southeast-1
# Output: note the SecretARN for use when creating the proxy
Creating the IAM Role for RDS Proxy
The proxy needs permission to read the secret from Secrets Manager.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowGetSecretValue",
"Effect": "Allow",
"Action": [
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret"
],
"Resource": "arn:aws:secretsmanager:ap-southeast-1:123456789:secret:prod/myapp/rds-credentials-*"
},
{
"Sid": "AllowDecryptWithKMS",
"Effect": "Allow",
"Action": "kms:Decrypt",
"Resource": "arn:aws:kms:ap-southeast-1:123456789:key/your-kms-key-id",
"Condition": {
"StringEquals": {
"kms:ViaService": "secretsmanager.ap-southeast-1.amazonaws.com"
}
}
}
]
}
Creating the RDS Proxy via AWS CLI
# Create the RDS Proxy
aws rds create-db-proxy \
--db-proxy-name "myapp-prod-proxy" \
--engine-family POSTGRESQL \
--auth '[{
"AuthScheme": "SECRETS",
"SecretArn": "arn:aws:secretsmanager:ap-southeast-1:123456789:secret:prod/myapp/rds-credentials-AbCdEf",
"IAMAuth": "DISABLED"
}]' \
--role-arn "arn:aws:iam::123456789:role/rds-proxy-role" \
--vpc-subnet-ids subnet-aaa subnet-bbb subnet-ccc \
--vpc-security-group-ids sg-proxy-id \
--require-tls \
--idle-client-timeout 1800 \
--region ap-southeast-1
# Register the RDS instance as a proxy target
aws rds register-db-proxy-targets \
--db-proxy-name "myapp-prod-proxy" \
--db-cluster-identifiers "myapp-prod-cluster" \
--region ap-southeast-1
# Configure the connection pool on the target group
aws rds modify-db-proxy-target-group \
--db-proxy-name "myapp-prod-proxy" \
--target-group-name "default" \
--connection-pool-config '{
"MaxConnectionsPercent": 80,
"MaxIdleConnectionsPercent": 50,
"ConnectionBorrowTimeout": 120,
"SessionPinningFilters": ["EXCLUDE_VARIABLE_SETS"]
}' \
--region ap-southeast-1
The MaxConnectionsPercent: 80 parameter means the proxy will use at most 80% of the database’s max_connections. The remaining 20% is a buffer for administrative and monitoring connections. SessionPinningFilters: EXCLUDE_VARIABLE_SETS instructs the proxy not to pin when a SET statement occurs — this improves multiplexing efficiency for most applications.
Security Group Configuration
# Security group for RDS Proxy
# Inbound: from the Lambda security group, on the database port
aws ec2 authorize-security-group-ingress \
--group-id sg-proxy-id \
--protocol tcp \
--port 5432 \
--source-group sg-lambda-id \
--region ap-southeast-1
# Security group for RDS
# Inbound: from the RDS Proxy security group only (not directly from Lambda)
aws ec2 authorize-security-group-ingress \
--group-id sg-rds-id \
--protocol tcp \
--port 5432 \
--source-group sg-proxy-id \
--region ap-southeast-1
# CORRECT: Lambda → Proxy → RDS
# ANTI-PATTERN: Lambda → RDS directly (bypasses the proxy, no centralized pooling)
Go Implementation: Lambda with RDS Proxy
Proper Connection Initialization
On Lambda, there’s one important trick: initialize the database connection outside the handler function. This allows connections to be reused across invocations on the same instance (warm start).
package main
import (
"context"
"database/sql"
"fmt"
"log"
"os"
"time"
"github.com/aws/aws-lambda-go/lambda"
_ "github.com/lib/pq"
)
// db is declared at the package level — initialized once, reused across invocations
var db *sql.DB
func init() {
// init() is called once when the Lambda instance is first created (cold start)
// Connections created here will be reused by all invocations on the same instance
var err error
db, err = initDB()
if err != nil {
// If the DB can't be initialized, the Lambda can't serve any requests
// Better to fail fast than return an error on every invocation
log.Fatalf("Failed to initialize database connection: %v", err)
}
}
func initDB() (*sql.DB, error) {
// Use the RDS Proxy endpoint, not RDS directly
host := os.Getenv("DB_PROXY_ENDPOINT") // from the Lambda env var
port := os.Getenv("DB_PORT")
user := os.Getenv("DB_USER")
password := os.Getenv("DB_PASSWORD")
dbname := os.Getenv("DB_NAME")
dsn := fmt.Sprintf(
"host=%s port=%s user=%s password=%s dbname=%s sslmode=require",
host, port, user, password, dbname,
)
database, err := sql.Open("postgres", dsn)
if err != nil {
return nil, fmt.Errorf("sql.Open failed: %w", err)
}
// CORRECT: the pool on Lambda must be small — RDS Proxy is the main manager
// MaxOpenConns is small because RDS Proxy does the multiplexing
database.SetMaxOpenConns(2)
database.SetMaxIdleConns(1)
// A short ConnMaxLifetime ensures connections rotate regularly
// and don't become stale after a long Lambda instance idle period
database.SetConnMaxLifetime(5 * time.Minute)
// ConnMaxIdleTime: close idle connections that aren't used
// This is important so the proxy can reclaim connections to its pool
database.SetConnMaxIdleTime(1 * time.Minute)
// Verify a connection can be created
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if err := database.PingContext(ctx); err != nil {
return nil, fmt.Errorf("database ping failed: %w", err)
}
log.Printf("Database connection successful to: %s", host)
return database, nil
}
// ANTI-PATTERN: initializing the connection inside the handler
// func handler(ctx context.Context, event MyEvent) (string, error) {
// db, _ := initDB() // ✗ Creates a new connection on every invocation
// defer db.Close() // ✗ Closes the connection when done — no reuse
// // ...
// }
Handler with Context and Retry
type OrderRequest struct {
UserID int64 `json:"user_id"`
Amount float64 `json:"amount"`
Items []Item `json:"items"`
}
type OrderResponse struct {
OrderID int64 `json:"order_id"`
Status string `json:"status"`
}
func handler(ctx context.Context, req OrderRequest) (*OrderResponse, error) {
// Use the Lambda context for cancellation and timeout propagation
// Lambda automatically cancels this context when the function times out
order, err := createOrder(ctx, req)
if err != nil {
log.Printf("ERROR createOrder: %v", err)
return nil, fmt.Errorf("failed to create order: %w", err)
}
return &OrderResponse{
OrderID: order.ID,
Status: "created",
}, nil
}
func createOrder(ctx context.Context, req OrderRequest) (*Order, error) {
// Use a transaction for operations that need atomicity
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return nil, fmt.Errorf("BeginTx failed: %w", err)
}
defer tx.Rollback() // No-op if already committed
var orderID int64
err = tx.QueryRowContext(ctx,
"INSERT INTO orders (user_id, amount, status) VALUES ($1, $2, 'pending') RETURNING id",
req.UserID, req.Amount,
).Scan(&orderID)
if err != nil {
return nil, fmt.Errorf("insert order failed: %w", err)
}
for _, item := range req.Items {
_, err = tx.ExecContext(ctx,
"INSERT INTO order_items (order_id, product_id, quantity) VALUES ($1, $2, $3)",
orderID, item.ProductID, item.Quantity,
)
if err != nil {
return nil, fmt.Errorf("insert order_item failed: %w", err)
}
}
if err := tx.Commit(); err != nil {
return nil, fmt.Errorf("Commit failed: %w", err)
}
return &Order{ID: orderID}, nil
}
func main() {
lambda.Start(handler)
}
Using IAM Authentication (the Safer Option)
Instead of storing a database password in an environment variable, Lambda can use an IAM token to authenticate to RDS Proxy. This token is generated automatically and valid for 15 minutes.
import (
"context"
"fmt"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/feature/rds/auth"
)
func generateIAMToken(ctx context.Context, proxyEndpoint, region, dbUser string) (string, error) {
cfg, err := config.LoadDefaultConfig(ctx, config.WithRegion(region))
if err != nil {
return "", fmt.Errorf("failed to load AWS config: %w", err)
}
// BuildAuthToken creates an IAM authentication token valid for 15 minutes
authToken, err := auth.BuildAuthToken(
ctx,
fmt.Sprintf("%s:5432", proxyEndpoint),
region,
dbUser,
cfg.Credentials,
)
if err != nil {
return "", fmt.Errorf("failed to generate IAM token: %w", err)
}
return authToken, nil
}
// With IAM auth, Lambda doesn't need to know the database password at all
// The Lambda IAM policy only needs: rds-db:connect to the specific proxy resource
IAM authentication requires an IAM policy granting the rds-db:connect permission to the Lambda execution role, and the RDS Proxy configured with IAMAuth: REQUIRED. This approach is safer because there are no static credentials needing manual rotation.
Patterns for Other Architectures
RDS Proxy is a specific solution for Lambda + RDS on AWS. More general distributed systems have several other patterns solving similar problems.
PgBouncer for Self-Hosted PostgreSQL or ECS
If your infrastructure isn’t fully on Lambda or you need more control, PgBouncer is a very battle-tested open-source connection pooler for PostgreSQL.
flowchart LR
subgraph Services["Many Service Instances"]
S1[Service A\nInstance 1]
S2[Service A\nInstance 2]
S3[Service B]
S4[Service C]
end
subgraph Pooler["PgBouncer"]
PG["PgBouncer\nTransaction pooling mode\nPool size: 50"]
end
subgraph DB["Database"]
RDS[(PostgreSQL\nmax_connections: 100)]
end
S1 & S2 & S3 & S4 --> PG
PG --> RDS
note["Transaction mode:\nconnections returned to the pool\nafter each transaction finishes\nnot after the session ends"]PgBouncer supports three pooling modes:
Session mode — a connection is assigned to a client for the duration of the session. Most compatible with all PostgreSQL features but does the least multiplexing.
Transaction mode — a connection is assigned to a client only while a transaction is active, then returned to the pool. This is the most efficient mode and suits most web applications.
Statement mode — a connection is assigned per statement. Most aggressive but incompatible with multi-statement transactions.
For microservices on ECS or Kubernetes, deploy PgBouncer as a sidecar container or as a standalone service.
ProxySQL for MySQL
The PgBouncer equivalent for MySQL is ProxySQL — a more feature-rich proxy with query routing, read/write splitting, and more detailed monitoring.
flowchart LR
subgraph Apps["Microservices"]
A1[Service 1]
A2[Service 2]
A3[Service 3]
end
subgraph ProxySQL["ProxySQL"]
PS["ProxySQL\nQuery routing\nConnection pooling\nRead/Write splitting"]
end
subgraph MySQL["MySQL Cluster"]
PRI[(Primary\nWrite)]
REP1[(Replica 1\nRead)]
REP2[(Replica 2\nRead)]
end
Apps --> PS
PS -->|Write queries| PRI
PS -->|Read queries| REP1
PS -->|Read queries| REP2ProxySQL can automatically route queries based on whether it’s a SELECT (to replicas) or INSERT/UPDATE/DELETE (to the primary), without application-level changes.
Observability: What to Monitor
Correctly configured pooling without adequate observability is still dangerous — you won’t know when you’re approaching limits or when a problem is developing.
Metrics to monitor in CloudWatch:
RDS:
□ DatabaseConnections — total connections to the database
Alert: > 80% of max_connections
□ FreeableMemory — available memory
Alert: < 20% of total memory
□ CPUUtilization
Alert: > 80% for > 5 minutes
RDS Proxy:
□ DatabaseConnectionRequests — connection requests to the proxy
□ DatabaseConnections — active connections from the proxy to RDS
□ DatabaseConnectionsCurrentlySessionPinned — pinned sessions
(High = there may be an optimizable pattern)
□ QueryDatabaseResponseTime — database response latency
□ ClientConnections — connections from Lambda to the proxy
Lambda:
□ Duration — if it rises drastically, there may be connection waiting
□ Errors — error spikes could indicate connection failures
□ Throttles — Lambda throttled due to the concurrency limit
# Query the RDS connections metric via AWS CLI
aws cloudwatch get-metric-statistics \
--namespace AWS/RDS \
--metric-name DatabaseConnections \
--dimensions Name=DBInstanceIdentifier,Value=myapp-prod \
--start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%S) \
--end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
--period 60 \
--statistics Average Maximum \
--region ap-southeast-1
# Query the RDS Proxy pinned sessions metric
aws cloudwatch get-metric-statistics \
--namespace AWS/RDS \
--metric-name DatabaseConnectionsCurrentlySessionPinned \
--dimensions Name=ProxyName,Value=myapp-prod-proxy \
--start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%S) \
--end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
--period 60 \
--statistics Average \
--region ap-southeast-1
If theDatabaseConnectionsCurrentlySessionPinnedmetric is consistently high, the proxy can’t multiplex optimally. Check whether the application usesSETstatements, prepared statements in a way that causes pinning, or very long transactions. Every pinned session is an exclusive physical connection — it can’t be shared.
When to Use What
USE RDS Proxy if:
✓ Lambda + RDS (this is its main use case)
✓ Many short-lived service instances connecting to RDS
✓ Very spiky, unpredictable traffic
✓ Need transparent failover without code changes
✓ Already in the AWS ecosystem and want a managed solution
USE PgBouncer if:
✓ Self-hosted PostgreSQL or in containers (ECS, Kubernetes)
✓ Need full control over the pooler configuration
✓ Want to avoid the extra RDS Proxy cost
✓ The team is already familiar with PostgreSQL administration
USE ProxySQL if:
✓ MySQL and need automatic read/write splitting
✓ Need pattern-based query routing
✓ Need detailed query-level observability
DON'T NEED A PROXY if:
✗ A monolith with a small, predictable number of instances
✗ Services with correctly configured pools
and a capped instance count
✗ CI/CD jobs or batch processing that aren't concurrent
Summary
- Classic pooling fails in distributed systems because its assumptions collapse: the instance count is unpredictable, lifecycles are short, and there’s no “global pool” that can be shared across instances.
- Lambda + RDS without a proxy is a time bomb — during traffic spikes, hundreds of instances each open connections and the database is exhausted within seconds.
- Reducing
MaxOpenConnson Lambda isn’t enough — it reduces the problem’s severity but doesn’t eliminate the uncertainty of the total connection count, because that depends on how many instances AWS creates.- RDS Proxy moves pooling to the infrastructure — one centralized pool serving all Lambda instances, with an explicitly controllable connection limit to the database.
- Initialize connections outside the handler, not inside — this enables connection reuse across warm invocations and saves cold start overhead.
- The pool on Lambda must be small —
MaxOpenConns: 1–2is enough because RDS Proxy does the multiplexing. A large pool on Lambda actually wastes the proxy’s multiplexing potential.SessionPinningFilters: EXCLUDE_VARIABLE_SETSreduces unnecessary pinning and improves multiplexing efficiency for most applications that don’t need session-level state.- Monitor
DatabaseConnectionsCurrentlySessionPinned— this metric shows how effectively multiplexing is running. High values indicate potential optimization at the application level.