Load Balancers & Reverse Proxies
A Load Balancer (LB) distributes incoming network traffic evenly across multiple backend servers to maximize throughput, minimize latency, optimize resource utilization, and eliminate single points of failure.
1. Why Use Load Balancers?
flowchart LR
subgraph Without["Without Load Balancer"]
U1["👥 Users"] --> SS["🖥️ Single Server\n(Overloaded! SPOF)"]
end
subgraph With["With Load Balancer"]
U2["👥 Users"] --> LB["⚖️ Load Balancer"]
LB --> A1["🖥️ App Server 1 (33%)"]
LB --> A2["🖥️ App Server 2 (33%)"]
LB --> A3["🖥️ App Server 3 (34%)"]
end
Core Benefits:
- High Availability & Redundancy: Automatically redirects traffic away from failing servers.
- Horizontal Elasticity: Add or remove server instances dynamically based on current load.
- Security & SSL Termination: Offloads CPU-intensive TLS encryption handshakes from application nodes.
2. Layer 4 vs Layer 7 Load Balancing
Load balancers operate at different layers of the OSI model:
| Characteristic | Layer 4 (Transport Layer) | Layer 7 (Application Layer) |
|---|---|---|
| Inspection Data | Source/Dest IP address & TCP/UDP Port | Full HTTP headers, Cookies, URL paths, Request Body |
| Routing Decision | Fast packet-level forwarding | Context-aware content routing |
| Processing Speed | Ultra-high performance, minimal CPU overhead | Slightly higher latency (must assemble and parse payload) |
| SSL Offloading | Passes encrypted packets through | Terminates SSL / inspects plaintext request |
| Use Case Examples | Raw TCP connections, database pooling, VoIP | Microservice routing (/api/v1/auth vs /api/v1/feed) |
| Tech Examples | AWS Network Load Balancer (NLB), HAProxy (TCP mode), IPVS | AWS Application Load Balancer (ALB), Nginx, Envoy, Traefik |
3. Load Balancing Algorithms
| Algorithm | How It Operates | Best Workload Scenario |
|---|---|---|
| Round Robin | Cycles requests sequentially: Server 1 → 2 → 3 → 1 | Identical hardware with equal task processing times |
| Weighted Round Robin | Allocates higher request ratios to more powerful nodes | Heterogeneous server clusters |
| Least Connections | Forwards to the server with fewest active TCP connections | Long-lived persistent requests (WebSockets, large downloads) |
| Least Response Time | Routes to the server with lowest latency & active connections | Latency-critical transactional APIs |
| IP Hash (Consistent Hashing) | Hashes client IP to assign deterministic server nodes | Session caching / sticky user routing without centralized store |
| Random with Two Choices | Picks two random servers and selects the less loaded one | Simple distributed load balancing preventing thundering herd |
4. Health Checks & Automatic Failover
Load balancers continuously execute health check probes against registered backend instances:
flowchart LR
LB["⚖️ Load Balancer"] -->|"GET /healthz"| S1["✅ Server 1: 200 OK\n(Kept in pool)"]
LB -->|"GET /healthz"| S2["✅ Server 2: 200 OK\n(Kept in pool)"]
LB -->|"GET /healthz"| S3["❌ Server 3: 500 ERROR\n(Removed from pool)"]
- Active Health Checks: The load balancer sends synthetic periodic requests (e.g., every 5 seconds) to
/healthz. - Passive Health Checks: The load balancer monitors real production traffic for connection errors and timeouts.
5. Reverse Proxy vs Load Balancer
| Feature | Reverse Proxy | Load Balancer |
|---|---|---|
| Primary Goal | Intermediary for security, caching, SSL, and request transformation | Distributes incoming traffic across 2+ backend targets |
| Target Scale | Can sit in front of even a single origin server | Requires a cluster of multiple target instances |
| Edge Caching | Caches static assets & responses directly | Typically passes requests straight to targets |
| Popular Tools | Nginx, Cloudflare, Apache | HAProxy, AWS ALB/NLB, Google Cloud Armor LB |
Note: Modern tools like Nginx and Envoy function simultaneously as both reverse proxies and application load balancers.