Load Balancer Routing: Mermaid Flowchart Diagram

About Source

A load balancer is a network component that distributes incoming requests across multiple backend servers, preventing any single server from becoming a bottleneck while providing fault tolerance and horizontal scalability.

Load balancers operate at different network layers. Layer 4 (Transport) load balancers route based on IP and TCP/UDP port — they see source/destination addresses but not HTTP content. Layer 7 (Application) load balancers parse HTTP headers and can route based on URL path, hostname, cookies, and request content. AWS ALB, Nginx, HAProxy, and Envoy operate at Layer 7.

Routing Algorithms:

- Round Robin: Requests are distributed sequentially across backends. Simple, effective when all backends are equivalent. - Weighted Round Robin: Backends with higher capacity receive proportionally more requests. - Least Connections: New requests go to the backend with the fewest active connections — better for variable-duration requests like file uploads. - IP Hash: Client IP is hashed to consistently route a given client to the same backend (session affinity / sticky sessions). - Random with Two Choices (P2C): Randomly sample two backends and pick the one with fewer connections — approaches optimal performance at scale.

Health Checks: Load balancers continuously probe backends (HTTP GET /health, TCP connect, or custom checks) at configurable intervals. Backends that fail health checks are removed from rotation until they recover. This is the primary mechanism for zero-downtime rolling deployments.

SSL/TLS Termination: L7 load balancers typically terminate TLS, forwarding plain HTTP to backends on an internal network. This centralizes certificate management and reduces backend CPU overhead.

See Reverse Proxy Request Flow for the closely related pattern where a single reverse proxy sits in front of backends.

Frequently asked questions

A load balancer distributes incoming requests across multiple backend servers to prevent any single server from becoming a bottleneck. It improves both fault tolerance (if one backend fails, traffic shifts to healthy ones) and scalability (add backends to increase capacity without changing client configuration).

Round robin distributes requests sequentially and works well when backends are equivalent. Least connections sends new requests to the backend with the fewest active connections — better for long-lived or variable-duration requests. IP hash consistently routes a given client IP to the same backend (sticky sessions) for session-stateful applications. Weighted variants of round robin and least connections account for differing backend capacities.

Layer 4 load balancers route based on IP and TCP/UDP port, without inspecting application content — fast but limited. Layer 7 load balancers parse HTTP headers and can route based on URL path, hostname, cookies, or custom headers, enabling advanced routing like path-based microservice routing and A/B testing. AWS ALB, Nginx, and Envoy operate at Layer 7.

Not configuring health checks (or using checks too infrequent to detect failures quickly), sticky sessions without a fallback when the target backend goes down, misconfigured TLS between the load balancer and backends, and not accounting for connection draining (abruptly removing a backend mid-request) are the most common issues.

Every load balancer is a reverse proxy, but not every reverse proxy is a load balancer. A reverse proxy may route to a single backend (for TLS termination, caching, or security filtering), while a load balancer's defining feature is distributing traffic across multiple backends for scalability and redundancy. In practice, production reverse proxies like Nginx and Envoy implement both functions.

mermaid

flowchart TD
    Client([Incoming Request]) --> LB[Load Balancer\nL4 or L7]

    LB --> HC{Health Check\nAll Backends}
    HC -->|Unhealthy| Remove[Remove from pool\nuntil recovery]
    HC -->|Healthy| Algo{Routing\nAlgorithm}

    Algo -->|Round Robin| B1[Backend Server 1]
    Algo -->|Round Robin| B2[Backend Server 2]
    Algo -->|Round Robin| B3[Backend Server 3]
    Algo -->|Least Connections| LCB[Backend with\nfewest active connections]
    Algo -->|IP Hash| StickyB[Consistent backend\nfor this client IP]

    B1 --> Resp([Response to client])
    B2 --> Resp
    B3 --> Resp
    LCB --> Resp
    StickyB --> Resp

    LB --> TLS[TLS Termination\ncertificate at LB]
    TLS --> BackendHTTP[Forward plain HTTP\nto backend pool]