What is an embedding in a RAG system?

An embedding in a RAG system is a numerical vector that represents the meaning of a piece of content, such as a document chunk or a user query.

Instead of storing only the original text for retrieval, a RAG system also converts that text into a vector produced by an embedding model.

Simple idea

Suppose a document contains:

The refund window is 30 days.

An embedding model converts that text into a vector such as:

[0.12, -0.44, 0.83, 0.07, ...]

The real vector may contain hundreds or thousands of numerical dimensions.

Those numbers are not human-readable words. Together, they encode patterns learned by the embedding model about the content's meaning.

What gets embedded in RAG?

A typical RAG system creates embeddings for:

• Document chunks during ingestion
• User queries during retrieval

For example:

Document chunk
    ↓
Embedding model
    ↓
Document vector

and:

User query
    ↓
Embedding model
    ↓
Query vector

The query vector can then be compared with stored document vectors.

Embeddings represent semantic information

Embeddings are designed so that pieces of text with similar meaning can be located near each other in vector space.

For example, these two phrases may receive similar representations:

How long can I request a refund?

and:

The refund window is 30 days.

They do not use the same words, but they express related meaning.

Embeddings are not the original content

An embedding is a mathematical representation of the content, not a replacement for the source text.

A RAG system normally stores both:

• The original chunk text
• Its embedding vector

It may also store metadata such as document ID, page number, section heading, or permissions.

Where are embeddings stored?

Document embeddings are commonly stored in a vector-capable database or search index.

A simplified record might look like:

chunk_id: C102
text: The refund window is 30 days.
embedding: [0.12, -0.44, 0.83, ...]
document_id: policy-7
page: 4

During retrieval, the system compares the query embedding with these stored vectors to find semantically related chunks.

Embedding dimension

Every embedding produced by a particular model has a fixed number of dimensions.

For example, conceptually:

Embedding = [dimension 1, dimension 2, ... dimension N]

The exact number depends on the embedding model.

All vectors being compared in the same vector space must be compatible.

Embeddings are model-dependent

Different embedding models learn different vector spaces.

A vector produced by one model should not normally be compared directly with vectors produced by an unrelated model.

This is why a RAG system should use compatible embedding models for indexed documents and incoming queries.

That topic is covered separately in “Why should document and query embeddings use compatible embedding models?”.

Embeddings can represent more than text

Depending on the model, embeddings can also represent:

• Images
• Audio
• Code
• Multimodal content

But in a standard text RAG pipeline, embeddings usually represent text chunks and user queries.

The reasons RAG systems rely on these vectors for retrieval are covered separately in “Why are embeddings used in RAG?”.

In short, an embedding is a dense numerical representation of content that places semantically related items near one another in vector space. In RAG, embeddings are created for document chunks and queries so the retrieval system can compare their meaning mathematically.