Circuit Breaker Pattern: Mermaid State Diagram

About Source

The circuit breaker pattern is a resilience mechanism that detects when a downstream service is failing and stops sending it requests — "opening" the circuit — to prevent cascading failures from propagating through a distributed system.

What the diagram shows

This state diagram models the three states a circuit breaker transitions between:

- Closed (normal operation): requests flow through freely. The breaker counts failures within a rolling window. Each successful call resets the failure counter. - Open (fault isolated): when the failure count exceeds the configured threshold (e.g., 5 failures in 10 seconds), the breaker trips open. All subsequent requests fail immediately without ever reaching the downstream service — a technique called "fail fast." This protects the downstream from being hammered while it recovers. - Half-Open (probe state): after a configurable timeout (e.g., 30 seconds), the breaker transitions to Half-Open and allows a single test request through. If the test request succeeds, the circuit closes and normal operation resumes. If it fails, the breaker returns to Open for another timeout cycle.

Why this matters

Without a circuit breaker, a slow or failing downstream service causes callers to pile up waiting for timeouts — exhausting thread pools and connection pools in the caller, which then starts failing its callers. The failure cascades upward. The circuit breaker breaks this chain by failing fast at the first sign of consistent downstream failure.

Netflix's Hystrix library popularized this pattern, and it is now built into service meshes like Istio and Linkerd. In practice, circuit breakers work best alongside Request Retry Logic — retries handle transient failures, while circuit breakers handle sustained outages. The Bulkhead Pattern complements circuit breakers by isolating thread pools so that one failing dependency doesn't exhaust resources for others.

Frequently asked questions

The circuit breaker pattern is a resilience mechanism that detects when a downstream service is consistently failing and stops sending it requests — opening the circuit — to prevent cascading failures. Named after an electrical circuit breaker, it protects the broader system by failing fast rather than waiting for slow timeouts to propagate upstream.

The breaker operates in three states. In the Closed state, requests flow normally while failures are counted within a rolling window. When failures exceed a threshold, the breaker trips to the Open state and all calls fail immediately without reaching the downstream service. After a timeout, the breaker enters Half-Open and allows a single probe request: success closes the circuit, failure returns it to Open for another timeout cycle.

Use a circuit breaker on any synchronous call to a dependency that could fail or become slow — external APIs, downstream microservices, and databases. It is most critical in microservices architectures where a single slow service can exhaust thread pools and trigger cascading failures across multiple services in the call chain.

Retry logic handles transient, one-off failures by re-attempting the same request after a short delay — it assumes the failure is temporary. The circuit breaker handles sustained, ongoing outages by stopping requests entirely after a failure threshold is crossed — it assumes retrying would make things worse. The two patterns are complementary: use retries for brief hiccups and circuit breakers to stop hammering a service that is genuinely down.

Setting the failure threshold too low (e.g., 1 failure opens the circuit) causes false positives that interrupt service unnecessarily. Setting the timeout too long delays recovery detection. Teams also frequently forget to emit metrics on circuit state transitions — without alerting on open circuits, outages go unnoticed until user complaints surface.

mermaid

stateDiagram-v2
    [*] --> Closed

    Closed --> Closed : Request succeeded
    Closed --> Closed : Failure count below threshold
    Closed --> Open : Failure threshold exceeded

    Open --> Open : Requests rejected with fail-fast error
    Open --> HalfOpen : Reset timeout elapsed

    HalfOpen --> Closed : Test request succeeded
    HalfOpen --> Open : Test request failed

    Closed --> [*] : Service decommissioned