MemGPT: Towards LLMs as Operating Systems
Charles Packer12 min readResearch
FOUNDATIONALAI AgentsAdvanced
Key takeaway
Foundational work on long-term contextual memory.
Updated on Oct 3, 2026
01
Abstract
- MemGPT treats an LLM agent as an operating system that manages its own limited context window.
- The paper introduces virtual context management so agents can work with documents and conversations larger than the model context.
- Main memory is the LLM's immediate context window.
- External context is managed through archival memory and other storage.
- The agent can explicitly move information between memory tiers using function calls.
- Paging and interrupts are inspired by operating-system memory management.
- The design targets long-term conversational agents and document analysis.
- MemGPT is evaluated on document question answering and conversational memory tasks.
02
Introduction
- Fixed context windows create a fundamental bottleneck for long-running LLM agents.
- Simply truncating old messages can discard information that becomes important later.
- Stuffing an entire long document into the prompt can exceed context limits and increase inference cost.
- Traditional retrieval systems usually decide what to retrieve outside the language model.
- MemGPT instead lets the LLM itself manage movement between memory tiers.
- The operating-system analogy provides a framework for deciding what stays in fast main memory and what is stored externally.
- Long-term memory is therefore treated as a resource-management problem as well as a retrieval problem.
03
Problem
- The problem is how to maintain coherent agent behavior when relevant information exceeds the model's finite context window.
- The agent needs to decide when to recall information from external memory.
- It also needs to decide when information in the current context should be archived.
- Memory operations must be exposed through an interface the LLM can invoke.
- The system must preserve important facts across long conversations.
- Document tasks require access to information distributed across contexts larger than a single prompt.
- The paper asks whether virtual context management can improve long-context behavior without changing the underlying LLM.
04
Methodology
- The agent receives a system prompt describing its memory hierarchy and available functions.
- Core memory contains the persistent persona and currently relevant conversation state.
- Archival memory stores larger amounts of text outside the context window.
- The agent can insert information into archival memory.
- The agent can search archival memory when relevant information is not present in core memory.
- The agent can edit or replace core-memory blocks.
- Paging operations are selected through normal LLM tool/function calls.
- The model observes function results and continues reasoning from the updated context.
- This creates a closed loop between reasoning and memory management.
Figure 1
05
Architecture
- MemGPT's architecture contains an LLM controller, core memory, archival memory, and function interfaces.
- Core memory includes editable blocks that can persist across messages.
- Archival memory is an external vector-like storage for large amounts of information.
- Memory-management functions include operations for storing and retrieving information.
- Interrupt handling enables the agent to react to external messages or events.
- The system prompt teaches the model when and why to use memory operations.
- The architecture is explicitly designed to decouple effective context size from the model's fixed context window.
- The LLM serves simultaneously as task solver and memory manager.
06
Dataset
- The paper evaluates MemGPT on two primary classes of tasks: document analysis and conversational memory.
- The document experiments use the SQuAD benchmark as a source of long documents and questions.
- The conversational experiments construct long-term user-agent interactions with information distributed across many turns.
- The paper also uses a synthetic conversation setup to evaluate whether facts can be retained over long interaction histories.
- Document evaluation tests retrieval of information outside the immediate context.
- Conversational evaluation tests whether the agent can recall user-specific facts after many intervening messages.
- The benchmarks are designed to stress context limits rather than conventional supervised training performance.
07
Training
- MemGPT is primarily an inference-time architecture and does not require fine-tuning the underlying LLM.
- The model is prompted with a system message explaining its memory-management functions.
- Memory contents are updated through tool calls generated during inference.
- Archival retrieval is invoked only when the model decides that relevant information is outside core memory.
- Core-memory edits allow persistent facts to remain available across subsequent turns.
- The architecture therefore adds an external memory hierarchy around an existing LLM.
- The experiments demonstrate the approach using GPT-4 and other supported language models rather than training a new memory-specialized model.
08
Experiments
- Document question answering evaluates whether MemGPT can access information beyond its immediate context.
- The system is tested under contexts larger than the underlying model's effective prompt capacity.
- Conversational experiments insert facts earlier in a dialogue and query them much later.
- The evaluation compares memory-managed conversations with context-window and retrieval baselines.
- The paper studies both automatic task accuracy and qualitative agent behavior.
- Long-context experiments test whether the model can selectively page information instead of keeping everything in active context.
- The experiments also demonstrate interruption handling and persistent persona memory.
09
Baselines
- Baselines include conventional context-window approaches that keep recent messages in the prompt.
- Retrieval-augmented approaches provide an external comparison for accessing long documents.
- The paper also compares against models with larger native context windows where applicable.
- A key conceptual baseline is naive truncation, which loses older conversation history.
- MemGPT differs from fixed retrieval because the LLM itself decides when memory should be written or read.
- The comparison therefore tests both effective context length and autonomous memory management.
10
Results
- MemGPT enables an LLM to operate over information larger than its native context window.
- In document QA, the system can retrieve relevant information from archival memory instead of requiring the entire document in context.
- In conversational settings, MemGPT can recall information from much earlier interactions.
- Persistent core-memory blocks preserve important persona and user information across turns.
- The operating-system abstraction provides a unified mechanism for short-term and long-term context.
- The paper demonstrates that memory operations can be learned through prompting rather than model retraining.
- The results support virtual context as a practical design pattern for long-running LLM agents.
11
Conclusion
- MemGPT reframes long-context agents as systems programmers managing a virtual memory hierarchy.
- Core memory provides fast access to currently relevant state.
- Archival memory provides persistent capacity beyond the context window.
- Paging and interrupts make memory management part of the agent's action space.
- The LLM can decide what to remember, what to retrieve, and what to keep active.
- This enables long-term conversation and document analysis without retraining the foundation model.
- The paper's broader contribution is an operating-system abstraction for building persistent LLM agents.
12
References
- Primary paper: Charles Packer et al., “MemGPT: Towards LLMs as Operating Systems,” arXiv:2310.08560.
- Authors include Charles Packer, Vivian Fang, Shishir Patil, Kevin Lin, Sarah Wooders, and Joseph E. Gonzalez.
- The paper introduces virtual context management for LLMs.
- Primary source: https://arxiv.org/abs/2310.08560.
- The work's central analogy is an operating system managing a finite main-memory context plus larger external storage.
- Experiments focus on document analysis and long-term conversational memory.
Source trail
Primary papers, implementation pages, and external source records used to ground this explainer.
Continue reading
Related research
AI Agents
GitSwarm
GitSwarm turns inference-time work into a persistent, branchable body of reusable contributions that later agents can build on.
AI AgentsLongHarness
LongHarness helps agents reason over 100M-token document collections by keeping evidence and conclusions as explicit, revisable states that guide one another.
AI AgentsGenerative Agents
An agent architecture combining memory, retrieval, reflection, and planning to simulate believable long-horizon behavior.