What is chunk overlap in RAG?

Chunk overlap in RAG is the amount of content intentionally repeated between two neighboring chunks.

It is used to reduce the chance that important information is lost when a document is split at a chunk boundary.

Simple example

Suppose the chunk size is 500 tokens and the overlap is 50 tokens.

The chunks may look like:

Chunk 1 → tokens 1–500
Chunk 2 → tokens 451–950
Chunk 3 → tokens 901–1400

The last 50 tokens of one chunk are repeated at the beginning of the next chunk.

Why overlap is useful

Important information often spans a boundary.

For example:

The access token expires after 30 minutes and
must then be refreshed using the refresh token.

If the split happens between those two parts, each chunk may contain only part of the statement.

Overlap increases the chance that at least one chunk contains enough of the complete idea to be retrieved and understood.

What overlap helps preserve

Overlap can help keep together:

• A sentence and its continuation
• A heading and the first paragraph below it
• A definition and its explanation
• Consecutive steps in a process
• Conditions and their outcomes

Overlap is not a separate chunking strategy

Overlap is usually a configuration applied to a chunking method.

It can be used with:

• Fixed-size chunking
• Recursive chunking
• Sentence-based chunking
• Some structure-based strategies

Too little overlap

If overlap is too small, important context may still be split across neighboring chunks.

This is more likely when:

• Sentences are long
• Concepts span several paragraphs
• Fixed-size boundaries are used

Too much overlap

Excessive overlap creates duplicated content in the index.

This can cause:

• More chunks to store
• More embeddings to generate
• Higher ingestion cost
• Similar chunks appearing together in search results
• Less diversity in the retrieved evidence

For example, if neighboring chunks are mostly identical, the retriever may return several versions of the same passage.

Overlap should not compensate for poor chunking

If natural document boundaries are available, it is usually better to preserve them rather than relying on very large overlap.

For example, keeping a paragraph intact is generally cleaner than splitting it arbitrarily and then duplicating half of it into the next chunk.

Sentence overlap

Overlap does not always have to be measured in tokens.

A sentence-based strategy might repeat one complete sentence:

Chunk 1 → Sentences 1, 2, 3
Chunk 2 → Sentences 3, 4, 5

This can preserve cleaner context than repeating an arbitrary token range.

How much overlap should be used?

There is no universal percentage that works for every RAG system.

The right amount depends on:

• Chunk size
• Document structure
• Typical sentence length
• Query complexity
• Retrieval quality
• How often important information crosses boundaries

Test several overlap values using representative queries and check whether retrieval improves without creating excessive duplication.

Signs overlap may be too high

Look for:

• Nearly identical chunks dominating top results
• High index size relative to source content
• Increased embedding cost with little retrieval improvement
• Repeated evidence consuming prompt space

For the broader chunk-size tradeoff, see “What is chunk size in RAG?”.

In short, chunk overlap repeats a small amount of neighboring content so information crossing a chunk boundary is less likely to be lost. It can improve context continuity, but excessive overlap creates duplication and can reduce retrieval diversity.