What is a circuit breaker pattern in multi-agent systems?

A circuit breaker is a reliability pattern that temporarily stops sending requests to an agent or dependency that is repeatedly failing. Instead of letting every new task hit an unhealthy service, the orchestrator fails fast, queues work where appropriate, or selects a safe fallback. This protects the rest of the multi-agent system from cascading failures.

Beginner-friendly example

Imagine an API test-generation workflow with three agents:

• Requirements agent: Extracts the endpoint specification.
• Security agent: Generates authentication and authorization test cases.
• Coverage agent: Reviews the final suite.

The security agent depends on an external model API. Its last 10 requests include 6 failures, a 60% failure rate. Suppose the system's configured rule is to open the circuit after at least 10 requests with a failure rate of 50% or higher.

The circuit opens: new requests to that dependency stop temporarily. The orchestrator can route eligible security tasks to an approved backup, preserve them for later processing, or report that security coverage is unavailable. It must not silently claim the test suite is fully reviewed.

Three circuit breaker states

1. Closed: Requests are allowed. Recent failures and timeouts are counted.
2. Open: Requests are blocked quickly for a configured cooldown period; callers use a fallback or receive a clear error.
3. Half-open: After the cooldown, a small, controlled number of probe requests are allowed. Successful probes can close the circuit; renewed failures reopen it.

Closed → Failure threshold reached → Open → Cooldown → Half-open → Close or reopen

A numerical walkthrough

Suppose a security agent normally responds in 2 seconds, but an upstream provider begins timing out after 15 seconds.

• Without a circuit breaker, 20 requests can each consume up to 15 seconds of waiting and occupy concurrency slots.
• With a breaker opening after the configured failure threshold, later requests fail fast or take an approved fallback route rather than repeatedly waiting.
• After a 30-second cooldown, the system permits a small probe rather than releasing the entire backlog at once.

These thresholds and timings are examples, not universal settings. Tune them to real service objectives and traffic patterns.

How do you implement it?

1. Define the protected boundary: Track failures per agent, model provider, endpoint, or tenant where isolation matters.
2. Classify failures: Count transient infrastructure errors, relevant timeouts, and server failures; avoid treating every invalid user request as proof the dependency is unhealthy.
3. Use a rolling window: Combine a minimum request count with a failure-rate or consecutive-failure threshold.
4. Open and share state: If several workers use the same dependency, coordinate breaker state or use a deliberately consistent per-instance policy.
5. Choose safe fallbacks: Reassign only to an authorized agent with compatible skills and contracts, or return an explicit partial/failed status.
6. Probe recovery carefully: Limit half-open traffic and avoid simultaneous recovery surges.
7. Monitor outcomes: Measure opens, probe success, failed calls avoided, fallback usage, and deadline misses.

Circuit breaker vs. retry and health check

• A retry attempts the same operation again to recover from a temporary failure.
• A health check assesses whether an agent or service seems available.
• A circuit breaker changes the routing policy after repeated failures, protecting the system until recovery is plausible.

Retries should be bounded and should respect an open circuit; otherwise they can worsen an outage. A healthy process heartbeat does not mean its dependent model API is functioning.

LangGraph and Celery example

In LangGraph, a routing node can inspect circuit state before invoking a specialist or its tool. If open, it can choose an explicitly approved fallback branch or produce a structured unavailable result.

For Celery, tasks executed by different workers may share one rate-limited provider. A circuit breaker must account for that shared dependency. Preserve task IDs and completion status when deferring or rerouting work. An open circuit should not be interpreted as successful execution.

In short

Circuit breakers prevent repeated calls to failing agents or dependencies, limit cascading failures, and allow controlled recovery. Combine them with monitoring, bounded retries, backpressure, and safe fallback policies.