How do you optimize memory-compaction scheduling to reduce bottlenecks in multi-agent systems?

Optimizing memory-compaction scheduling means deciding when compaction jobs should run, how much worker capacity they may use, and which jobs deserve priority so summarization does not block the agents it is meant to help. The key is to keep compaction off the user-visible critical path where possible while still completing it before a workflow would exceed its safe context budget.

Unlike choosing a *trigger threshold* for a single workflow, scheduling looks across many workflows competing for workers, rate limits, and storage throughput.

Beginner-friendly example: multi-agent API test generation

A system uses LangGraph for requirements extraction, test generation, security review, and publication. Celery workers run 50 concurrent workflows. Each workflow may request a compaction after accumulating tool results.

If all 50 compactions arrive at once and share a worker queue with security reviews, compaction can delay higher-priority review tasks. Even though the resulting prompts are shorter, the entire test-generation service may become slower.

Forecast demand → Classify urgency → Choose queues and worker limits → Coalesce duplicate jobs → Compact ahead of handoffs → Monitor lag and quality → Adapt scheduling

Numerical example: avoiding a compaction queue

Suppose 60 workflows each request one compaction taking an average of 6 seconds of worker execution, and the service has 3 dedicated compaction workers.

1. Total work is 60 × 6 = 360 worker-seconds.
2. With perfectly balanced, uninterrupted execution, the theoretical minimum processing time is 360 / 3 = 120 seconds.
3. If all requests arrive together and each worker processes 20 jobs sequentially, the last job finishes after approximately 120 seconds; its wait before starting is roughly 114 seconds.
4. During that wait, some workflows may hit context-budget limits. Adding more retries to the same queue would make congestion worse.
5. If 40 requests are duplicates or low-value because the same source revision has already been compacted, coalescing them leaves 20 distinct jobs.
6. At the same assumed service time and concurrency, processing 20 jobs takes roughly ceil(20/3) × 6 = 42 seconds in balanced waves, without counting real-world overhead.

The savings come primarily from removing redundant work, not from assuming that queueing itself improves model speed. Actual service times vary; monitor p95 job duration and external model rate limits rather than treating these estimates as guarantees.

What scheduling strategies work?

1. Separate compaction capacity from critical agents: Use a dedicated Celery queue or bounded concurrency so background summaries cannot consume every worker needed for validation or publication.
2. Classify urgency: Prioritize workflows projected to exceed their next-step context budget. Compactions for workflows with ample room can be lower priority.
3. Precompute at safe boundaries: Compact after a stable checkpoint or before a known expensive handoff, ideally while another independent task can proceed.
4. Coalesce duplicate requests: Use a key such as tenant_id + workflow_id + source_revision + projection_role. One validated result can serve several equivalent requests when permissions and context requirements match.
5. Debounce bursts: Wait briefly for rapid updates to settle if the workflow has sufficient context headroom. Do not debounce past a hard context safety deadline.
6. Bound queue and rate-limit pressure: Apply per-tenant fairness, caps on concurrent model calls, and backoff with jitter for transient rate limits. Do not let one tenant monopolize the compactor.
7. Schedule by expected value: A summary reused by ten upcoming agents can be more valuable than one produced for a workflow one step from completion.
8. Use incremental compaction where safe: Summarize only newly committed events and merge them with a verified version, while periodically checking against authoritative sources to avoid accumulated summarization errors.
9. Handle overload explicitly: Fall back to targeted source retrieval, smaller authorized context, or a blocked step if no validated summary can be produced before the limit.
10. Keep source revisions in every job: A stale or late compaction task must not activate a summary that omits newer agent updates.

Example: priority score

For illustration, classify candidates using three signals:

• Context urgency: How close the next model call is to the usable token limit.
• Expected reuse: How many future agent calls benefit.
• Workflow criticality: Whether the next consumer is blocked without a safe summary.

A task that must compact before the very next security review is higher priority than a nightly cache refresh. Use an explicit upper bound on waiting time and fair-share rules so low-priority work is not starved indefinitely. A numeric scoring formula is optional; clear priority classes plus aging often work well.

Queueing versus critical-path latency

The compaction service's execution time and queue wait time are different measurements.

A compaction may take 6 seconds to execute but wait 90 seconds in the queue. If the next agent cannot proceed without that summary, the workflow experiences nearly 96 seconds of compaction-related delay. If the compaction completes while unrelated branches are running and is ready before the next handoff, most of that work may not add to end-to-end latency.

For parallel LangGraph branches, measure the critical path, not the sum of all task durations.

Safe schedule-aware fallback

Before each model call, check the projected token budget and the availability of a validated summary covering the required source revision. If a compaction job is queued but not complete, do not treat its pending candidate as usable.

If the safe context fits, proceed with the last validated summary plus the complete new-event tail. If it does not fit, use scoped retrieval or delay/block the affected call. Never silently omit mandatory authorization rules or unresolved approvals to maintain throughput.

LangGraph and Celery implementation

In LangGraph, create a compaction task at a stable node boundary and persist its source revision and intended consumer. A downstream node can check whether the validated summary exists; if not, it chooses a safe fallback path. Checkpoint persistence does not itself guarantee that the background summary has finished.

In Celery, route compaction tasks to a distinct queue with explicit worker concurrency, retries, deadlines, and deduplication keys. For long tasks, understand task acknowledgment and redelivery behavior; make activation idempotent and conditional on the source revision. Avoid over-reserving tasks if that prevents urgent jobs from reaching available workers. Capacity plans must account for shared LLM API rate limits even when local Celery concurrency is increased.

What should you measure and tune?

• Compaction queue depth, queue age, p50/p95 wait and execution times.
• Number of coalesced/skipped jobs, retries, rate-limit errors, and stale results.
• Percentage of workflows blocked waiting for compaction.
• Context-overflow failures and safe-fallback usage.
• End-to-end p95 completion time, cost per accepted workflow, and output quality.
• Per-tenant fairness and starvation incidents.

A rollout should compare these before and after changing scheduling policy. Faster compaction throughput is not a success if it degrades security-review quality or starves higher-priority work.

Difference from previous lessons

• Q180–181 and Q198: General bottlenecks, backpressure, and scheduling fairness.
• Q230: When one workflow should request compaction.
• Q231: How concurrent compaction results remain consistent.
• Q236: How to measure compaction's cost and latency.
• This lesson: How to schedule compaction work across many workflows using capacity isolation, prioritization, coalescing, and overload controls.

In short

Run compaction early enough to protect future context budgets, but isolate and prioritize it so it never overwhelms critical agent work. Measure queue wait, critical-path impact, fairness, and correctness—not merely how many summaries per minute the workers produce.

Further reading

• Celery: Routing Tasks
• Celery: Optimizing
• LangGraph: Persistence