How do you choose when to trigger memory compaction in a long-running multi-agent workflow?

Trigger memory compaction when the next multi-agent step is likely to exceed a safe context budget, when accumulated history becomes inefficient to reuse, or at a stable handoff boundary—provided the essential facts can be preserved. Waiting until the model rejects an oversized prompt is too late; compacting after every message can waste tokens and repeatedly degrade information.

The key distinction is when to compact, not how to summarize or score a summary. A good trigger policy combines measured context pressure, workflow checkpoints, expected future growth, compaction cost, and the risk of losing critical detail.

Beginner-friendly example: API test generation

A LangGraph workflow uses a requirements agent, test generator, security reviewer, and publisher to create tests for a large API. The first agent reads requirements, the second accumulates test drafts and tool results, and the reviewer adds findings. Each handoff can repeat thousands of tokens.

Compaction becomes useful before an expensive next step that would otherwise reread old logs. But a security review should not begin with an unverified summary that omits the OAuth scope, failed test IDs, or pending approvals.

Measure current context → Reserve space for the next node → Check trigger and safety gates → Compact at a stable boundary → Validate → Reuse the new summary

Numerical example: choose a safe threshold

Assume the model supports a 32,000-token context window. A production policy reserves:

• 4,000 tokens for system instructions and tool definitions.
• 5,000 tokens for the next retrieved documents and tool results.
• 3,000 tokens for the expected output and uncertainty.
• 2,000 tokens as an operational safety margin.

The maximum budget available for previously accumulated conversation and working memory is therefore 32,000 − 4,000 − 5,000 − 3,000 − 2,000 = 18,000 tokens.

1. Current history: 15,000 tokens. Next step is expected to add 2,000. Projected history is 17,000: stay below the budget; compaction may be skipped.
2. Later history: 17,000 tokens. Next step may add 4,000. Projected history is 21,000, exceeding the 18,000 safe budget by 3,000. Trigger compaction *before* that step.
3. A validated summary plus recent unsummarized messages takes 7,000 tokens. With the expected 4,000 extra, the projected history is 11,000 tokens, leaving 7,000 of the history budget.
4. If compaction costs 3,000 tokens and avoids replaying 10,000 tokens in each of the next 5 agent calls, the illustrative gross repeated-context reduction is 50,000 tokens. After the compaction cost, the approximate net token reduction is 47,000 tokens, ignoring caching, fallback retrieval, and different pricing for inputs versus outputs.

These values are examples, not recommended universal thresholds. Use the exact model and tool limits that apply to the deployment. Token estimates can be inaccurate for tool outputs, multimodal content, and nonstandard message formatting.

Trigger strategies and when to use them

1. Absolute token threshold: Compact when active messages exceed a measured token count. Predictable, but the same number may be inappropriate for models with different context windows.
2. Fraction of context window: Trigger near a configured share of usable input capacity, after reserving room for tools, retrieval, and output. This adapts better across models.
3. Projected-next-step threshold: Trigger when current_tokens + expected_next_step_tokens > safe_history_budget. This helps prevent a failure before a large tool result or handoff.
4. Message-count threshold: Useful as a cheap early signal, but 20 short messages and 20 large tool responses have very different sizes.
5. Workflow-boundary trigger: Compact after requirements extraction, at a validated checkpoint, or before a branch join or agent handoff. These boundaries offer stable data and clearer invariants.
6. Cost-benefit trigger: Compact when expected future repeated-context savings outweigh summary-generation cost and risk. A task with one step left may not benefit.
7. Quality or relevance trigger: If retrieval quality degrades because earlier messages dominate the prompt, consider a targeted reduction even before the token limit.
8. Emergency guard: If the projected context is too large and safe compaction fails, use selective authorized retrieval, smaller inputs, or stop with an explicit error instead of silently dropping required evidence.

A practical policy often combines a projected token budget and a safe checkpoint boundary. Do not delay a necessary context reduction past the model limit just to reach a preferred boundary.

Add hysteresis to prevent repeated compaction

A naive policy might compact at 18,000 tokens, produce a 17,500-token summary, and trigger again almost immediately. This causes extra model calls and repeated summarization loss.

Instead, use hysteresis: a high trigger, a substantially lower post-compaction target, and a minimum interval or minimum amount of new content before another compaction.

Example: trigger when projected working context exceeds 18,000 tokens, but aim to reduce it to 8,000 tokens. Preserve the last recent messages and source pointers as part of that 8,000. The next compaction is considered only after meaningful new content accumulates or a hard safety threshold is approached.

Which moments should not trigger compaction?

• During unresolved tool-call pairs: Keep a model's tool invocation and corresponding tool response consistent, rather than leaving an orphaned call in the compact context.
• During active external side effects: Do not restructure context mid-publication or while a non-idempotent tool operation is pending.
• Before safeguarding source evidence: Archive required raw references and maintain a recovery path.
• When critical facts cannot pass validation: Do not publish a summary that loses approval state, authorization constraints, source versions, or open issues.
• When there is insufficient expected reuse: Compaction overhead may exceed savings for a workflow about to finish.

Separate full durable records from the context actually supplied to the model. Trimming the latter does not require deleting the former.

Example operational policy

A coordinator can implement the following logic at each model-call boundary:

PYTHON
projected = current_context_tokens + expected_next_step_tokens

if projected > safe_context_budget:
    if checkpoint_is_safe and source_revision_is_current:
        candidate = create_compact_context(source_state)
        if verify_required_facts(candidate, source_state):
            activate_if_revision_matches(candidate)
        else:
            use_targeted_source_retrieval_or_block()
    else:
        use_targeted_source_retrieval_or_block()
else:
    continue_without_compaction()

The pseudocode leaves out production-specific error handling and synchronization. A real system must also record the source revision, tenant, candidate version, evaluation result, and active-summary pointer.

LangGraph and Celery integration

In LangGraph, place a dedicated summarization or context-preparation node before expensive model calls or after stable milestones. If using LangMem's SummarizationNode, distinguish max_tokens_before_summary (when summarization begins) from max_tokens (the result budget), and tune token counting for the deployed model. A checkpoint persists graph state; it does not mean all historical messages should be sent to every model.

In LangChain agent middleware, summarization triggers can use token counts, message counts, or fractions of context size. These are configurable thresholds, not automatic guarantees that critical information will survive. Keep required structured state outside the compressed conversational text.

With Celery, schedule expensive compaction as a retry-safe job keyed by workflow ID and source revision. Before activating the result, verify that no newer agent update invalidated it. If the job is delayed, enforce a synchronous context-budget guard at the next model-call boundary; do not blindly proceed with an oversized context.

What should be monitored?

Track the number of compactions per workflow, tokens before and after, percentage of compactions accepted, generation cost, added latency, projected-context-overflow incidents, fallback retrieval calls, and downstream quality regressions. An excessive compaction rate can indicate poor target sizing; overflow errors suggest triggers are too late.

Difference from earlier lessons

• Q201: Allocates token budgets across collaborating agents.
• Q227: Implements the compaction mechanism and version-safe activation.
• Q228: Preserves important facts and constraints.
• Q229: Evaluates quality and efficiency.
• This lesson: Specifies trigger timing, predictive thresholds, hysteresis, workflow boundaries, and when compaction should be skipped.

In short

Trigger before the next step threatens the usable context budget, preferably at a safe workflow boundary, and avoid repeated compaction with a lower post-compaction target. Use measured growth, expected reuse, and validation gates—not a fixed message count alone.

Further reading

• LangMem: How to manage long context with summarization
• LangMem: Short-term memory API reference
• LangChain: Built-in agent middleware