How do you prioritize critical memory-compaction requests during system overload?

During system overload, prioritize memory-compaction requests by how urgently a validated summary is needed for a safe next agent step—not simply by which job entered the queue first. The coordinator must preserve capacity for critical agent work, avoid tenant starvation, and never promote an incomplete summary to satisfy a deadline.

This lesson focuses on priority assignment and fairness under an already overloaded system. Queue admission and backpressure are separate mechanisms.

Beginner-friendly production incident

A LangGraph pipeline has requirements, test-generation, security-review, and publication agents. Celery compaction workers receive a burst of summary requests after an API specification is uploaded.

Two jobs are waiting:

• Job A: The security reviewer cannot proceed because the next prompt would exceed its safe context budget in 20 seconds. The required OAuth scopes and pending approval must be preserved.
• Job B: A completed workflow requests an optional historical summary for later analytics; it has no immediate consumer.

A FIFO queue might process Job B first. Priority-aware scheduling favors Job A, but only if there is enough time to produce and validate the summary; otherwise, the reviewer uses a verified fallback or pauses safely.

Classify next-step criticality → Calculate time-to-deadline → Check context safety → Reserve fair capacity → Dispatch → Validate → Activate or fall back

Step-by-step numerical example

Suppose one available compaction worker can finish and validate a job in approximately 10 seconds. Four jobs arrive together:

1. A — Security-review handoff: Needed in 15 seconds; otherwise the prompt would exceed its approved budget.
2. B — Test-generator handoff: Needed in 40 seconds; safe verified context is available for now.
3. C — Routine analytics: Needed in 10 minutes, with no blocked workflow.
4. D — Another tenant's security-review handoff: Needed in 25 seconds.

A possible order is A → D → B → C.

• A completes at approximately 10 seconds, before its 15-second deadline.
• D completes at approximately 20 seconds, before its 25-second deadline.
• B completes at approximately 30 seconds, before its 40-second deadline.
• C completes at approximately 40 seconds, well before its 10-minute deadline.

This works under the idealized fixed-duration assumptions. In real deployments, task duration varies, and a job's deadline includes queue wait + model execution + validation + activation. An already running Celery task is not safely preempted just because a higher-priority job arrives.

How should a priority score be designed?

Use a small number of explainable signals rather than arbitrary labels:

1. Deadline slack: slack = time_remaining − estimated_queue_and_execution_time. Lower positive slack means the job is more urgent; negative slack may mean the required summary cannot be delivered on time.
2. Blocked workflow: Give additional priority when a critical agent is actually waiting and has no safe alternative context.
3. Risk of context overflow: Compaction becomes urgent when the projected next model input exceeds its verified usable budget.
4. Downstream importance: A pending authorization review or required human-approval handoff matters more than an optional summary for offline reporting.
5. Expected reuse: A valid summary needed by several immediate next nodes can provide more value, provided it does not starve urgent one-off work.
6. Source readiness: Prefer jobs whose source revision is stable enough that the summary is likely to remain valid; avoid repeatedly prioritizing jobs invalidated by active writes.
7. Tenant fairness: Apply per-tenant quotas, weighted fair sharing, and aging so a noisy tenant cannot monopolize the queue.

Do not confuse priority with permission to bypass validation. Security-critical summaries often require stronger validation, which must be included in the time estimate.

Suggested priority classes

• Critical: A safe model call cannot proceed without a validated summary or another safe recovery path, and the deadline is close.
• High: A summary is required at an upcoming handoff, but some headroom remains.
• Normal: Beneficial compaction for running workflows without immediate context pressure.
• Background: Maintenance, optional analytics, and speculative context precomputation.

Use deadline ordering or aging *within* classes and reserve a limited share of service capacity for lower priorities to prevent indefinite starvation. Class names are operational conventions, not inherent model properties.

What if a deadline cannot be met?

Imagine Job A has 5 seconds before a required handoff, but generation and validation normally need 10 seconds. Making it the highest priority does not create extra capacity.

The coordinator should:

• Use the last validated summary plus a complete authorized event tail, if it fits.
• Retrieve only the necessary source requirements and approval evidence, if that fits the model budget.
• Delay or pause the affected node at a safe checkpoint when no complete trustworthy context is available.
• Avoid using a partial summary, dropping important facts, or treating pending approval as granted.

It may still be worthwhile to schedule a later compaction for future calls, but not to claim that it met an impossible deadline.

Fairness under sustained overload

Strict priority can starve routine tasks indefinitely. For example, if urgent work continuously fills the queue, low-priority maintenance may never run.

A fair scheduler can:

• Reserve a bounded portion of worker/API capacity for noncritical work.
• Increase a task's effective priority as its waiting age grows.
• Cap each tenant's concurrent compaction jobs.
• Coalesce equivalent jobs so multiple agents do not consume separate capacity for identical revisions.
• Expire jobs after their intended consumer has passed, finished, or moved to a new source revision.

Fairness must not override hard security gates or allow another tenant's data into an agent's compact context.

Celery implementation details

Use dedicated queues for urgent versus background compaction, with explicitly allocated workers or broker-supported priorities where available. Celery priority behavior depends on the broker and is not identical across Redis and RabbitMQ configurations. A high-priority message may still wait if every worker has already reserved long-running low-priority tasks.

Tune prefetch according to execution time; isolating urgent workers may help more than relying only on broker priority. Tasks should carry workflow ID, tenant ID, source revision, priority class, deadline, and logical deduplication key.

Priorities should be reassessed when a job's source revision becomes stale or its consumer no longer needs the summary. A stale compaction task must not overwrite newer validated context.

LangGraph implementation details

At each handoff, calculate whether the next node can run using validated memory within its context budget. If not, request a compaction with an explicit consumer and deadline, then persist the workflow state. When compaction finishes, verify the candidate's version and content before resuming.

LangGraph checkpoints help a workflow resume; they do not automatically prioritize Celery tasks or provide global queue fairness. Priority control belongs in the scheduling, admission, and worker architecture.

Metrics to monitor

Track deadline-miss rate by class, p95 queue age, critical-agent blocked time, percentage of critical jobs meeting validation gates, priority inversion events, fairness by tenant, starvation age, unnecessary compactions, stale candidates, and downstream task correctness.

An apparent speedup is not acceptable if skipped validation leads to missing authorization tests. Analyze impact on end-to-end accepted workflow latency and cost, not merely worker throughput.

Difference from nearby questions

• Q198: Prevents starvation in general multi-agent scheduling.
• Q237: Improves overall compaction scheduling and bottlenecks.
• Q238: Scales worker capacity.
• Q239: Limits incoming demand and establishes safe overload behavior.
• This lesson: Selects which admitted compaction requests receive scarce capacity first, using deadlines, context urgency, workflow importance, and tenant fairness.

In short

Prioritize compaction based on deadline slack and the next agent's need for a validated context, while reserving fair capacity and enforcing safety gates. If a critical deadline cannot be met, use verified fallback context or pause the workflow—never ship an unvalidated summary.

Further reading

• Celery: Routing Tasks
• Celery: Optimizing
• LangGraph: Persistence