What is the difference between exact search and ANN search?

Exact search and ANN search both try to find vectors that are closest to a query vector, but they differ in how much of the vector collection they inspect.

The main trade-off is:

Exact search maximizes search accuracy. ANN search maximizes speed and scalability.

Exact search

Exact nearest-neighbor search compares the query vector against every eligible vector in the collection.

Then it calculates the chosen similarity or distance metric and returns the true nearest Top-K results.

For example:

Query → compare with all 1,000,000 vectors → sort by similarity → return exact Top-5

Because every candidate is checked, exact search does not lose recall because of indexing shortcuts.

ANN search

Approximate Nearest Neighbor search avoids comparing the query with every vector.

Instead, it uses an index such as HNSW or IVF to quickly locate promising candidates.

For example:

Query → navigate ANN index → inspect a much smaller candidate set → return approximate Top-K

This dramatically reduces search latency, but the index may occasionally miss a true nearest neighbor.

Example

Assume the true nearest vectors are:

1. A
2. B
3. C
4. D
5. E

Exact search returns:

A, B, C, D, E

An ANN search might return:

A, B, D, C, F

The ANN result is still very close, but E was missed and F appeared instead.

Which one is faster?

ANN is usually much faster for large vector collections because it avoids exhaustive comparisons.

Exact search becomes increasingly expensive as:

• Number of vectors grows
• Embedding dimension grows
• Query traffic increases

For small collections, however, exact search may already be fast enough.

Which one has better recall?

Exact search provides the true nearest neighbors for the chosen distance metric.

So, from a vector-search perspective, its nearest-neighbor recall is effectively the reference baseline.

ANN can have slightly lower recall because it searches only part of the space.

How much recall it loses depends on:

• Index type
• Index parameters
• Search parameters
• Dataset structure

Why ANN is common in RAG

A production RAG system may contain millions or tens of millions of chunks.

Scanning every vector for every query can make retrieval too slow.

ANN gives a practical balance:

High retrieval recall + low latency + better scalability

That is usually more useful than mathematically perfect search that is too expensive to serve.

Why exact search is still important

Exact search is useful for:

• Small datasets
• Offline evaluation
• Creating an ANN baseline
• Measuring ANN Recall@K
• Debugging retrieval quality

For example, if exact search retrieves the correct evidence but ANN does not, the issue is likely related to the ANN index or its configuration rather than the embedding model alone.

ANN often works with reranking

A common RAG design is:

ANN → retrieve 20–50 candidates → reranker → select best 5 → send context to LLM

ANN provides speed and broad recall.

Reranking provides a more precise ordering of the smaller candidate set.

Interview takeaway

Exact search compares the query against every eligible vector and returns the true nearest neighbors, giving maximum search recall but increasing latency as the dataset grows. ANN searches only promising candidates using an index, making retrieval much faster and more scalable at the cost of occasionally missing a true nearest neighbor. In production RAG, ANN is usually preferred at scale, while exact search is valuable for small datasets and as an evaluation baseline.