Where is knowledge stored in an LLM?

An LLM does not store knowledge in a database-like list of facts. Most of what it learns during training is encoded across its learned parameters.

Those parameters are the billions of numerical weights inside components such as:

• Token embeddings
• Self-attention layers
• Feed-forward networks
• Output layers

Knowledge is distributed

A fact or concept is usually not stored in one single parameter.

Instead, information is represented across many parameters and model components working together.

For example, the model's ability to complete:

The capital of France is ...

with:

Paris

comes from patterns learned during training and distributed throughout the network.

There is usually no single location that can be pointed to as:

parameter 8,241,931 = "Paris is the capital of France"

Parameters store learned patterns

During training, the model repeatedly predicts tokens and adjusts its parameters when its predictions are wrong.

Over time, these updates encode patterns involving:

• Language structure
• Word relationships
• Factual associations
• Common reasoning patterns
• Writing styles
• Code structures

These learned patterns are sometimes called the model's parametric knowledge.

What about the current prompt?

Not everything the model uses comes from its parameters.

When an LLM processes a prompt, it creates temporary internal representations called activations or hidden states.

So it is useful to distinguish:

Parameters → knowledge learned during training
Context / activations → information available in the current prompt

For example, if you tell the model:

Our internal project is called Aurora.

the model can use that information during the current interaction even if it was never part of its original training.

Can an LLM retrieve external knowledge?

Yes, an application can also connect an LLM to external sources such as databases, search systems, tools, or retrieval-augmented generation (RAG).

That information is not stored inside the model's parameters. It is supplied to the model as additional context when needed.

Is knowledge perfectly stored in the parameters?

No.

The model learns statistical patterns rather than maintaining a perfectly structured knowledge base. As a result, knowledge can be incomplete, approximate, outdated, or incorrectly recalled.

For the underlying mechanism, see “What are parameters in an LLM?”

In short: most knowledge learned during LLM training is distributed across the model's parameters, while prompt-specific and externally retrieved information is provided through the model's current context.