How does IVF work?

IVF works by first dividing the vector space into clusters, then searching only the clusters that are most likely to contain the nearest vectors.

The goal is to avoid scanning the entire vector collection for every query.

1. Train cluster centroids

During index creation, IVF learns a set of representative cluster centers, usually with a clustering algorithm such as k-means.

For example:

1,000,000 embeddings → learn 1,000 centroids

Each centroid represents one region of the vector space.

2. Assign each vector to a cluster

Every stored vector is compared with the centroids and assigned to its nearest one.

So the index becomes a set of inverted lists:

Centroid 1 → vectors assigned to cluster 1

Centroid 2 → vectors assigned to cluster 2

Centroid 3 → vectors assigned to cluster 3

and so on.

These lists are what IVF searches later.

3. Embed the user query

At retrieval time, the user's question is converted into a query embedding using the same compatible embedding space as the stored vectors.

4. Find the nearest centroids

Instead of immediately searching all stored vectors, IVF first compares the query with the cluster centroids.

It selects the centroids closest to the query.

For example:

1,000 total clusters → query is closest to clusters 17, 204, and 611

5. Search only selected clusters

IVF then compares the query with vectors inside those selected inverted lists.

So instead of scanning 1 million vectors, the system might search only a much smaller candidate set.

This is where most of the speedup comes from.

6. Return the Top-K matches

The candidate vectors are scored using the configured similarity or distance metric.

The closest Top-K vectors are returned to the RAG pipeline.

The flow becomes:

Query → query embedding → nearest centroids → selected inverted lists → candidate vector search → Top-K chunks

What controls how many clusters are searched?

A parameter commonly called nprobe determines how many inverted lists are searched for each query.

For example:

nprobe = 1 → search only the nearest cluster

nprobe = 10 → search the 10 nearest clusters

Higher nprobe usually means:

• Better recall
• More vectors examined
• Higher latency

Lower nprobe usually means:

• Faster search
• Less computation
• Greater risk of missing relevant vectors

Why can IVF miss a relevant vector?

Consider a query located near the boundary between two clusters.

The true nearest document vector may be stored in the neighboring cluster.

If IVF searches only one cluster, that vector may never be considered.

Searching multiple nearby clusters reduces this problem.

Example in RAG

Suppose a knowledge base contains 5 million chunk embeddings divided into 5,000 IVF clusters.

A user asks:

"What is the refund period for enterprise customers?"

The query embedding is closest to a few clusters related to policies and customer terms.

IVF searches only those clusters, retrieves the strongest candidate chunks, and passes them to the next RAG stage.

IVF is a two-stage search

A useful way to remember IVF is:

Stage 1: coarse search Find the nearest cluster centroids.

Stage 2: fine search Search the vectors inside those selected clusters.

That is why IVF is often described as a coarse-to-fine retrieval method.

Interview takeaway

IVF first clusters the vector space and assigns every vector to an inverted list associated with a centroid. At query time, it finds the nearest centroids, searches only their lists, and returns the best Top-K candidates. This makes retrieval much faster than scanning the full collection, while nprobe controls the trade-off between recall and latency.