What is nprobe in IVF search?

In IVF search, nprobe controls how many inverted lists, or clusters, are searched for a query.

It is one of the most important parameters for balancing retrieval quality and speed.

How it works

Suppose an IVF index contains 1,000 clusters.

A query embedding is first compared with the cluster centroids.

If:

nprobe = 1

the search looks only inside the single nearest cluster.

If:

nprobe = 10

the search looks inside the 10 nearest clusters.

So nprobe determines how widely IVF searches around the query.

Why does nprobe matter?

IVF is approximate because it does not search every cluster.

A relevant vector may be stored in a nearby cluster rather than the single closest cluster.

If nprobe is too low, that cluster may never be searched.

Increasing nprobe gives the search more chances to find those relevant vectors.

Low nprobe

A low value means:

• Fewer clusters searched
• Fewer vector comparisons
• Lower latency
• Lower compute cost
• Greater risk of missing true nearest neighbors

This can work well when latency is the highest priority and the clusters are well separated.

High nprobe

A higher value means:

• More clusters searched
• More candidate vectors examined
• Better recall in many cases
• Higher latency
• More compute

As nprobe approaches the total number of IVF clusters, the search becomes less approximate because more of the vector space is examined.

Example

Assume the true answer-bearing chunk is stored in the third-nearest cluster to the query.

With:

nprobe = 1

IVF searches only the nearest cluster, so the correct chunk is missed.

With:

nprobe = 3

the third cluster is also searched, so the chunk can now be retrieved.

This is why nprobe directly affects Recall@K.

nprobe is not Top-K

These two settings control different things.

nprobe controls how many IVF clusters are searched.

Top-K controls how many final vectors are returned.

For example:

nprobe = 20 Top-K = 5

means:

• Search 20 nearby clusters
• Score candidate vectors from those clusters
• Return the best 5 results

Increasing Top-K does not help if the correct vector's cluster was never searched in the first place.

How should nprobe be chosen?

There is no universal best value.

Tune it using representative queries and measure:

• Recall@K
• Retrieval latency
• Throughput
• Resource usage

A practical process is:

Start with a small nprobe → increase it gradually → observe recall improvement → stop when the additional recall is no longer worth the extra latency

nprobe also depends on the number of clusters

An nprobe value should not be interpreted in isolation.

For example:

Searching 10 clusters out of 100 is very different from searching 10 clusters out of 10,000.

So the right value depends on:

• Total number of IVF lists
• Dataset size
• Vector distribution
• Latency target
• Required recall

RAG impact

If nprobe is too low, IVF may miss the answer-bearing chunk even when the embedding model represented it correctly.

If nprobe is unnecessarily high, retrieval may become slower without meaningfully improving answer quality.

So nprobe should be tuned as part of the RAG retrieval system, not treated as a fixed default.

Interview takeaway

nprobe is the number of IVF clusters searched for each query. A higher nprobe usually improves retrieval recall because more regions of the vector space are examined, but it also increases latency and compute. A lower nprobe is faster but increases the chance of missing relevant vectors. The right value should be chosen by benchmarking recall and latency on the actual RAG workload.