How are document chunks converted into embeddings?

Document chunks are converted into embeddings by passing each chunk through an embedding model, which transforms the chunk's text into a fixed-length numerical vector.

This usually happens during the RAG ingestion pipeline after documents have been parsed and split into chunks.

1. Start with a document chunk

After chunking, the system may have text such as:

The refund window is 30 days from the date of purchase.

The chunk may also have metadata such as:

• Document ID
• Page number
• Section heading
• Source URL
• Permissions

The metadata is usually stored alongside the embedding rather than being converted into the same vector automatically.

2. Prepare the text for the embedding model

The chunk text is formatted according to the embedding model's requirements.

This may include:

• Normalizing whitespace
• Preserving useful headings
• Adding selected contextual metadata to the text
• Ensuring the chunk fits within the model's input limit

For example:

Section: Refund Policy

The refund window is 30 days from the date of purchase.

Whether headings or other context should be included depends on the retrieval design.

3. Tokenize the chunk

The embedding model does not directly process raw characters.

The text is first converted into tokens by the model's tokenizer.

Conceptually:

Chunk text
   ↓
Tokenizer
   ↓
Token IDs

The exact tokenization depends on the embedding model.

4. Pass the tokens through the embedding model

The tokenized chunk is then processed by the embedding model.

Tokenized chunk
      ↓
Embedding model
      ↓
Vector representation

The model converts the semantic information in the chunk into a numerical representation.

5. Produce a fixed-length vector

The output is a vector with a fixed number of dimensions determined by the model.

For example:

[0.021, -0.118, 0.734, 0.055, ...]

A real embedding may contain hundreds or thousands of values.

Every chunk embedded with the same model produces a vector with the same dimensionality.

6. Store the vector with the chunk

The embedding is normally stored together with the original text and metadata.

A simplified record may look like:

chunk_id: C104
text: The refund window is 30 days from the date of purchase.
embedding: [0.021, -0.118, 0.734, ...]
document_id: policy-12
section: Refund Policy
page: 7

The embedding can be stored in a vector database or another system that supports vector indexing and similarity search.

7. Repeat this for every chunk

The ingestion pipeline processes all chunks in the knowledge base.

Document
   ↓
Chunks
   ↓
Embedding model
   ↓
Chunk vectors
   ↓
Vector index

Each vector represents one retrieval unit.

Embeddings are often generated in batches

For efficiency, production systems commonly send several chunks to the embedding model in one batch rather than embedding them one at a time.

Conceptually:

[Chunk 1, Chunk 2, Chunk 3, ...]
              ↓
        Embedding model
              ↓
[Vector 1, Vector 2, Vector 3, ...]

Batching can improve ingestion throughput while producing one embedding per chunk.

What happens later during retrieval?

When a user submits a query, the query is converted into an embedding using a compatible model.

The query vector is compared with the stored chunk vectors to identify semantically similar chunks.

The query-side process is covered separately in “How is a user query converted into an embedding?”.

Why model consistency matters

Document chunks and queries should be embedded into the same compatible vector space.

If document vectors were created with one unrelated embedding model and the query vector with another, similarity scores would generally not be meaningful.

This is covered in “Why should document and query embeddings use compatible embedding models?”.

In short, document chunks are converted into embeddings by preparing and tokenizing each chunk, passing it through an embedding model, receiving a fixed-length numerical vector, and storing that vector alongside the original chunk and its metadata for later similarity search.