How do you implement adaptive memory-compaction scheduling based on real-time workload conditions?

Adaptive memory-compaction scheduling is a feedback-control approach that continuously adjusts *when*, *how aggressively*, and *at what priority* a multi-agent system compacts its model-facing context based on observed load, predicted next-step context needs, queue delay, model-provider capacity, and quality risk.

A fixed policy might compact every 10,000 tokens. An adaptive controller instead asks whether a summary is needed before the next handoff, whether it can be completed safely, and whether it should run now or after load subsides. It does not change the canonical event history.

Beginner-friendly production incident

A LangGraph-based service uses a requirements agent, test generator, security reviewer, and publisher to build API test suites. Celery workers generate validated compact summaries.

At night, compaction queues are nearly empty, so the system can refresh summaries ahead of time. During a morning API-import burst, the LLM provider begins returning rate-limit errors. The old timer-based policy keeps sending optional compactions, starving the security reviewer.

An adaptive scheduler reduces speculative jobs, coalesces obsolete requests, reserves provider capacity for critical handoffs, and requests compaction only when a future agent's safe context budget requires it.

Observe metrics → Predict demand and deadlines → Select policy → Admit and prioritize bounded work → Validate result → Measure outcome → Adjust again

Step-by-step numerical example

Consider a compaction service with a sustainable capacity of 30 validated summaries per minute, including its shared LLM quota.

1. Normal load: 12 useful compaction requests/minute arrive and the p95 queue wait is 2 seconds. With plenty of headroom, the controller allows optional precompaction for workflows likely to need it later.
2. Burst begins: Demand rises to 42 requests/minute. Without controls, backlog grows by 42 − 30 = 12 jobs/minute.
3. After five minutes, backlog increases by about 60 jobs, assuming constant rates and no retries. Queue age and predicted handoff misses rise.
4. The controller enters a congested state: it disables optional refreshes worth 8 jobs/minute and coalesces 7 redundant revision-specific requests/minute.
5. Effective admitted demand falls to 42 − 8 − 7 = 27 jobs/minute.
6. Since capacity is 30/minute, existing backlog can shrink by about 3 jobs/minute. The accumulated 60-job backlog would take approximately 20 minutes to drain if those conditions persist.
7. Once queue age and quota headroom stay healthy for a sustained window, the controller gradually restores speculative work instead of immediately admitting all 15 suppressed jobs/minute.

The numerical rates are illustrative and assume the suppressed jobs are genuinely optional or redundant. Deferred *required* work cannot simply be dropped; it must remain traceable and receive capacity or a safe fallback.

Inputs to an adaptive scheduler

• Context headroom: Projected next model input relative to that role's usable token limit, after reserving tokens for tools and output.
• Time to next handoff: How soon a downstream agent needs the compacted context and whether validated fallback context is available.
• Queue pressure: Unique pending jobs, oldest queue age, p50/p95 wait, worker reservations, and successful completions.
• Provider headroom: Remaining request and token budget, 429 responses, retry-after signals, and global rate limiter state.
• Expected future reuse: Number of likely future model calls that can benefit from the summary.
• Job quality risk: Source revision churn, previous validation failures, unresolved security constraints, and likely need for exact source retrieval.
• Fairness: Tenant quotas, minimum progress for useful background tasks, and aging of jobs waiting unusually long.
• Cost: Expected avoided downstream tokens minus generation, validation, retrieval, and infrastructure overhead.

Avoid putting raw confidential memory content into scheduler metrics. Use appropriately scoped metadata and bounded counters.

A simple three-state controller

Healthy: Queues are short, validated throughput exceeds current admitted demand, and provider quota has headroom. Permit safe proactive compaction.

Congested: Queue age rises, quota headroom falls, or predicted handoff misses increase. Coalesce equivalent jobs, pause optional refreshes, allocate capacity by deadline slack, and retain a limited fair share for useful background work.

Critical: The next required context cannot be made ready before a hard deadline or rate limiting persists. Stop speculative compaction, protect urgent service capacity, and route affected agents to verified fallback or durable pause.

Use hysteresis: the threshold for leaving a congested state should be meaningfully healthier than the threshold for entering it, and should hold for a minimum observation window. Otherwise, the controller can rapidly flip between states and repeatedly start and cancel jobs.

For example, entering congestion after sustained p95 queue wait greater than 20 seconds and returning to healthy only after sustained wait less than 8 seconds avoids oscillation. These are example thresholds; tune them to real workflow deadlines and task durations.

What decision should the controller make for each request?

1. Check necessity: Will an upcoming model call overflow or lose mandatory context without a new validated summary?
2. Check reuse: Is a new summary likely to save enough future token cost to justify itself?
3. Check duplicates: Does an equivalent validated version or pending job already cover the same tenant, workflow, source watermark, and role?
4. Check feasibility: Can queue wait, generation, validation, and activation finish before the consumer needs it?
5. Check constraints: Are provider quota, tenant fairness, cost budget, and source revision requirements satisfied?
6. Select action: COMPACT_NOW, COMPACT_LATER, JOIN_EXISTING, USE_VERIFIED_FALLBACK, or PAUSE_SAFELY.
7. Recheck before activation: A summary generated from a stale revision cannot overwrite newer active context.

Example scheduling pseudocode

PYTHON
def decide(job, metrics, next_step):
    if validated_context_fits(next_step):
        return "USE_VERIFIED_CONTEXT"
    if equivalent_validated_or_pending(job):
        return "JOIN_EXISTING"
    if not can_finish_before_deadline(job, metrics):
        return "SAFE_FALLBACK_OR_PAUSE"
    if metrics.provider_throttled:
        return "DEFER_WITH_GLOBAL_QUOTA"
    if metrics.state == "CRITICAL" and not job.required_now:
        return "DEFER_OPTIONAL"
    return "ADMIT_WITH_PRIORITY"

This is illustrative decision logic, not a production-ready scheduler. The first branch must check current revision, required source evidence, permissions, and output token allowance—not just a token count. Admission and deduplication need atomic coordination when multiple schedulers operate concurrently.

Prevent feedback-loop failures

An adaptive controller can make things worse if it reacts only to the last few seconds of data.

• Oscillation: Rapidly switching scheduling modes creates bursts. Apply smoothing, hysteresis, and cooldowns.
• Retry amplification: Scaling admission after many failures can increase 429 responses. Count validated completions rather than attempted calls.
• Priority starvation: Continuously favoring urgent tasks prevents maintenance and eventually creates more urgent work. Use minimum-progress guards and tenant fairness.
• Stale metrics: Broker queue depth may exclude reserved tasks. Consider ready, reserved, executing, and delayed jobs.
• Unstable source revisions: Aggressive compaction during rapid agent writes wastes work. Debounce when the next handoff allows it.
• Bad prediction: A predicted deadline may be wrong. Recompute at actual handoffs and maintain safe fallback behavior.

LangGraph and Celery implementation

In LangGraph, emit or schedule compaction requests at stable graph boundaries with workflow ID, source watermark, consuming role, context headroom, and handoff deadline. Persist those metadata in checkpoint or durable task state. The downstream node verifies the summary is validated, authorized, and current enough before using it; otherwise it retrieves exact evidence or pauses.

In Celery, send admitted work to bounded queues with separate pools or capacity guarantees for urgent and background jobs. Collect worker queue-delay, execution, retry, and validated-completion measurements. An application-level controller can periodically update admission, weights, and priority classes. Broker priority behavior is transport-specific; worker concurrency and prefetch still affect the time an urgent job waits.

Scale worker replicas only when local capacity is the bottleneck. If provider tokens/minute are exhausted, more Celery workers do not create additional API quota.

How to verify adaptive scheduling works

Run comparable workloads through both fixed and adaptive scheduling:

• Correctness: No missing mandatory constraints, approvals, or committed events in consumed context.
• Timeliness: Lower critical handoff deadline-miss rate and p95/p99 end-to-end completion latency.
• Efficiency: Lower fully loaded compaction cost per accepted workflow and fewer stale or redundant summaries.
• Stability: Fewer 429s, retry storms, rapid state changes, and persistent queue spikes.
• Fairness: Background jobs and small tenants still achieve validated progress.
• Safety: Every failure uses a verified fallback or pauses; no unvalidated summary is activated.

Start with rule-based feedback before attempting learned prediction. A complicated predictor is not useful unless it beats a transparent baseline on held-out traffic and overload drills.

Difference from earlier lessons

• Q230: Decides when an individual long-running workflow should trigger compaction.
• Q236: Measures its cost and latency impact.
• Q237: Schedules jobs to reduce bottlenecks.
• Q238: Autoscale compaction workers.
• Q239: Applies overload admission and backpressure.
• Q240–242: Prioritization, priority inversion, and fair progress.
• This lesson: Closes the feedback loop, adapting trigger timing, admission, priorities, and optional work to changing demand while maintaining safety constraints.

In short

Adapt compaction policies to real workload signals, not one permanent token or queue threshold. Use stable feedback states, deadline-aware capacity planning, validated fallback, and measured outcomes so the scheduling controller improves latency and cost without risking multi-agent correctness.

Further reading

• Celery: Optimizing
• Celery: Routing Tasks
• LangGraph: Persistence