Research

MemGPT: Towards LLMs as Operating Systems

Charles Packer12 min readResearch
FOUNDATIONALAI AgentsAdvanced

Key takeaway

Foundational work on long-term contextual memory.

Updated on Oct 3, 2026

01

Abstract

  • MemGPT treats an LLM agent as an operating system that manages its own limited context window.
  • The paper introduces virtual context management so agents can work with documents and conversations larger than the model context.
  • Main memory is the LLM's immediate context window.
  • External context is managed through archival memory and other storage.
  • The agent can explicitly move information between memory tiers using function calls.
  • Paging and interrupts are inspired by operating-system memory management.
  • The design targets long-term conversational agents and document analysis.
  • MemGPT is evaluated on document question answering and conversational memory tasks.

02

Introduction

  • Fixed context windows create a fundamental bottleneck for long-running LLM agents.
  • Simply truncating old messages can discard information that becomes important later.
  • Stuffing an entire long document into the prompt can exceed context limits and increase inference cost.
  • Traditional retrieval systems usually decide what to retrieve outside the language model.
  • MemGPT instead lets the LLM itself manage movement between memory tiers.
  • The operating-system analogy provides a framework for deciding what stays in fast main memory and what is stored externally.
  • Long-term memory is therefore treated as a resource-management problem as well as a retrieval problem.

03

Problem

  • The problem is how to maintain coherent agent behavior when relevant information exceeds the model's finite context window.
  • The agent needs to decide when to recall information from external memory.
  • It also needs to decide when information in the current context should be archived.
  • Memory operations must be exposed through an interface the LLM can invoke.
  • The system must preserve important facts across long conversations.
  • Document tasks require access to information distributed across contexts larger than a single prompt.
  • The paper asks whether virtual context management can improve long-context behavior without changing the underlying LLM.

04

Methodology

  • The agent receives a system prompt describing its memory hierarchy and available functions.
  • Core memory contains the persistent persona and currently relevant conversation state.
  • Archival memory stores larger amounts of text outside the context window.
  • The agent can insert information into archival memory.
  • The agent can search archival memory when relevant information is not present in core memory.
  • The agent can edit or replace core-memory blocks.
  • Paging operations are selected through normal LLM tool/function calls.
  • The model observes function results and continues reasoning from the updated context.
  • This creates a closed loop between reasoning and memory management.
Figure 1

05

Architecture

  • MemGPT's architecture contains an LLM controller, core memory, archival memory, and function interfaces.
  • Core memory includes editable blocks that can persist across messages.
  • Archival memory is an external vector-like storage for large amounts of information.
  • Memory-management functions include operations for storing and retrieving information.
  • Interrupt handling enables the agent to react to external messages or events.
  • The system prompt teaches the model when and why to use memory operations.
  • The architecture is explicitly designed to decouple effective context size from the model's fixed context window.
  • The LLM serves simultaneously as task solver and memory manager.

06

Dataset

  • The paper evaluates MemGPT on two primary classes of tasks: document analysis and conversational memory.
  • The document experiments use the SQuAD benchmark as a source of long documents and questions.
  • The conversational experiments construct long-term user-agent interactions with information distributed across many turns.
  • The paper also uses a synthetic conversation setup to evaluate whether facts can be retained over long interaction histories.
  • Document evaluation tests retrieval of information outside the immediate context.
  • Conversational evaluation tests whether the agent can recall user-specific facts after many intervening messages.
  • The benchmarks are designed to stress context limits rather than conventional supervised training performance.

07

Training

  • MemGPT is primarily an inference-time architecture and does not require fine-tuning the underlying LLM.
  • The model is prompted with a system message explaining its memory-management functions.
  • Memory contents are updated through tool calls generated during inference.
  • Archival retrieval is invoked only when the model decides that relevant information is outside core memory.
  • Core-memory edits allow persistent facts to remain available across subsequent turns.
  • The architecture therefore adds an external memory hierarchy around an existing LLM.
  • The experiments demonstrate the approach using GPT-4 and other supported language models rather than training a new memory-specialized model.

08

Experiments

  • Document question answering evaluates whether MemGPT can access information beyond its immediate context.
  • The system is tested under contexts larger than the underlying model's effective prompt capacity.
  • Conversational experiments insert facts earlier in a dialogue and query them much later.
  • The evaluation compares memory-managed conversations with context-window and retrieval baselines.
  • The paper studies both automatic task accuracy and qualitative agent behavior.
  • Long-context experiments test whether the model can selectively page information instead of keeping everything in active context.
  • The experiments also demonstrate interruption handling and persistent persona memory.

09

Baselines

  • Baselines include conventional context-window approaches that keep recent messages in the prompt.
  • Retrieval-augmented approaches provide an external comparison for accessing long documents.
  • The paper also compares against models with larger native context windows where applicable.
  • A key conceptual baseline is naive truncation, which loses older conversation history.
  • MemGPT differs from fixed retrieval because the LLM itself decides when memory should be written or read.
  • The comparison therefore tests both effective context length and autonomous memory management.

10

Results

  • MemGPT enables an LLM to operate over information larger than its native context window.
  • In document QA, the system can retrieve relevant information from archival memory instead of requiring the entire document in context.
  • In conversational settings, MemGPT can recall information from much earlier interactions.
  • Persistent core-memory blocks preserve important persona and user information across turns.
  • The operating-system abstraction provides a unified mechanism for short-term and long-term context.
  • The paper demonstrates that memory operations can be learned through prompting rather than model retraining.
  • The results support virtual context as a practical design pattern for long-running LLM agents.

11

Conclusion

  • MemGPT reframes long-context agents as systems programmers managing a virtual memory hierarchy.
  • Core memory provides fast access to currently relevant state.
  • Archival memory provides persistent capacity beyond the context window.
  • Paging and interrupts make memory management part of the agent's action space.
  • The LLM can decide what to remember, what to retrieve, and what to keep active.
  • This enables long-term conversation and document analysis without retraining the foundation model.
  • The paper's broader contribution is an operating-system abstraction for building persistent LLM agents.

12

References

  • Primary paper: Charles Packer et al., “MemGPT: Towards LLMs as Operating Systems,” arXiv:2310.08560.
  • Authors include Charles Packer, Vivian Fang, Shishir Patil, Kevin Lin, Sarah Wooders, and Joseph E. Gonzalez.
  • The paper introduces virtual context management for LLMs.
  • Primary source: https://arxiv.org/abs/2310.08560.
  • The work's central analogy is an operating system managing a finite main-memory context plus larger external storage.
  • Experiments focus on document analysis and long-term conversational memory.

Source trail

Primary papers, implementation pages, and external source records used to ground this explainer.

Continue reading

Related research

Browse all research