How do you manage retention and deletion of compacted memory versions in multi-agent systems?

Managing retention and deletion of compacted-memory versions means deciding which historical summaries must remain available for active workflows, audits, or recovery, and which can be removed without breaking agents or exposing deleted information. Unlike ordinary memory TTL, compacted summaries can form a *version history*: one version may still be referenced by a running workflow, an evaluation result, or a rollback pointer.

The safest approach is version-aware retention, reference tracking, tombstones, and verified cleanup. Never delete a version solely because a newer summary exists.

Beginner-friendly example: API test-generation agents

A requirements agent, test generator, security reviewer, and publisher share a compacted history for POST /payments.

• Summary S1 includes events 1–100.
• Summary S2 includes events 1–180.
• Summary S3 includes events 1–240.
• A paused security-review workflow still references S2, while new workflows use S3.

Deleting S2 immediately could break the paused review or make its audit trace unreproducible. But retaining every summary forever increases storage cost and can retain personal or sensitive information longer than authorized.

Inventory versions and references → Apply retention and deletion rules → Block obsolete reads → Rebase active workflows → Purge eligible versions → Verify derived copies

Numerical example

Assume a project contains 1,000 historical summary versions:

1. 100 are current active versions.
2. 150 are superseded but still referenced by paused workflows or approved audit retention.
3. 50 are under an applicable preservation hold.
4. 700 are superseded, unreferenced, and past the approved retention period.

If the groups are mutually exclusive, the cleaner may mark 700 / 1,000 = 70% as deletion candidates. If each occupies 40 KB, eligible stored payload is 700 × 40 KB = 28,000 KB, approximately 28 MB in decimal units.

This does not mean 28 MB is immediately reclaimed: databases, snapshots, backups, indexes, and object-store retention have separate cleanup schedules. First confirm that no protected reference or policy prevents deletion.

Retention rules for versioned summaries

1. Identify the version: Track tenant, workflow/thread ID, summary ID, source-event watermark, creation time, and content hash.
2. Record references: Maintain the active-summary pointer, paused-checkpoint references, audit dependencies, rollback dependencies, and any explicit retention holds.
3. Define policies by purpose: Working summaries may have short operational retention; audit records may require longer retention. Apply current business, privacy, contractual, and legal requirements rather than assuming a universal period.
4. Separate content from provenance: Where permitted, preserve minimal non-sensitive lineage metadata without retaining full raw text indefinitely.
5. Use deletion states: Mark a version ACTIVE, SUPERSEDED, DELETE_PENDING, or DELETED with immutable audit events and a verified completion record.
6. Require safe dereferencing: Move or revalidate paused workflows before deleting their referenced versions; do not silently switch them to a newer summary with different meaning.
7. Handle sensitive deletion requests: Invalidate relevant summaries immediately in authorized reads, trace derived copies containing the information, and honor applicable deletion obligations and exceptions.
8. Clean derivatives: Remove or invalidate vector entries, cached snippets, materialized views, stored evaluations, and authorized exports where required and feasible.
9. Make cleanup idempotent: Stable summary IDs and generation/version checks stop retried tasks from accidentally deleting an unrelated replacement.
10. Verify and monitor: Check remaining live references, unexpected retrieval hits, cleanup failures, and backup lifecycle limitations.

Why ordinary reference counting may be insufficient

Suppose S2 has no direct active-summary pointer, but a paused LangGraph checkpoint still contains summary_id=S2. A cleaner that counts only currently active readers could delete S2 prematurely.

More subtle: an evaluation artifact might depend on S2 as its source, or an older summary might be needed for a documented rollback. Before removal, account for these durable references and any retention rule. A lease can protect short-lived readers, but it is not a substitute for a persistent reference from a paused workflow.

Reference count zero is a useful signal, not sufficient permission to delete. Expiration, holds, privacy rules, and downstream dependencies still apply.

What if a person requests deletion of information that appears in many summaries?

Deleting only the original memory record is insufficient if S1, S2, S3, and search indexes still contain the same sensitive text.

A safe process is:

• Identify source IDs and summary lineage that incorporated the data.
• Revoke or filter affected summaries from future retrieval while cleanup proceeds.
• Rebuild required current summaries from permitted remaining sources, or block affected workflows until safe.
• Purge eligible derived content and associated caches/index entries.
• Record minimal deletion evidence without copying the deleted content into the audit log.
• Account for backup retention and applicable legal preservation requirements.

Cryptographic hashes and references should be reviewed for privacy impact; a hash alone does not guarantee anonymity.

Safe cleanup under concurrency

At 10:00, a cleaner reads S2 as unreferenced. At 10:01, a workflow tries to pin S2. If these actions are not coordinated, the cleaner might delete a version that the workflow has just selected.

Use a transactional state transition: first mark S2 DELETE_PENDING with a generation check and prevent new references; then confirm existing references are absent before physical deletion. The workflow must atomically pin only a version still eligible for reading. If that fails, it must choose an authorized alternative and revalidate its required context.

Across a database and an external vector store, physical deletion is generally not one atomic transaction. Use durable cleanup jobs, tombstones, and read-time filters so a stale index cannot make deleted content available again.

LangGraph and Celery example

With LangGraph, record which compacted summary version each checkpoint consumed. A resumable thread may need that version even after it is superseded. Before deletion, verify that the thread has completed, been validly migrated, or no longer requires the content; comply with any overriding deletion policy.

With Celery, run deletion in idempotent bounded batches. Include tenant ID, summary ID, and version/generation in each task. Retry failed storage or index cleanup, and never allow a delayed cleanup task to remove a newly re-created version with a different generation.

What do you measure?

Useful metrics include number of superseded versions, protected references, eligible deletion candidates, oldest retained version age, deletion completion lag, orphaned cache/vector entries, blocked reads of tombstoned summaries, and failed cleanup retries. Audit the *decision* to retain or remove a version, not only the physical deletion event.

How does this differ from previous lessons?

• Q226 (expiration and garbage collection): When ordinary memory records expire and how their copies are cleaned up.
• Q233 (versioning and auditing compacted memory): How summary history and provenance are recorded.
• This lesson: How to safely retire particular historical summary versions while respecting paused workflow references, audit dependencies, deletion requests, and cross-store cleanup.

In short

Retain only the compacted versions still needed or legally required, block new references before deleting them, and verify cleanup across every derived store. Historical summary deletion is a dependency and privacy problem—not simply deleting the oldest row.