What is the difference between Flat, HNSW, IVF, PQ, and IVF-PQ indexes?

Flat, HNSW, IVF, PQ, and IVF-PQ are approaches used in vector search, but they solve different problems. Flat, HNSW, and IVF primarily describe how vectors are searched; PQ describes how vectors are compressed. IVF-PQ combines search-space pruning with compression.

1. Flat index — exhaustive search

A Flat index compares the query embedding with every stored vector and ranks them by the chosen similarity or distance metric.

• Strength: Exact Top-K nearest neighbors under that metric, assuming the same candidate set and full-precision scoring.
• Limitation: Work grows with the number of vectors, so latency and compute can become expensive at scale.
• Best fit: Small collections, accuracy-critical searches, or establishing an exact-search benchmark.

2. HNSW — graph-based approximate search

Hierarchical Navigable Small World (HNSW) connects vectors in a multilayer graph. Search follows promising neighbors instead of examining every vector.

• Strength: Often excellent recall at low query latency.
• Limitation: Graph links add memory overhead, and index construction can be expensive.
• Key settings: M, efConstruction, and query-time efSearch.
• Best fit: Interactive RAG workloads where fast responses and high recall matter and sufficient memory is available.

3. IVF — cluster-based approximate search

Inverted File Index (IVF) partitions vectors into groups around coarse centroids. At query time, it searches vectors in only the nearest selected groups.

• Strength: Reduces the number of candidates scored.
• Limitation: Relevant neighbors can be missed in clusters that were not searched.
• Key settings: Number of clusters (nlist) and clusters searched per query (nprobe).
• Best fit: Large datasets where scanning every vector is impractical.

Important: IVF alone does not necessarily compress the stored vector values; an IVF-Flat index can keep full-precision vectors.

4. PQ — compressed vector representation

Product Quantization (PQ) splits a vector into subvectors and represents each one using an index into a learned codebook.

• Strength: Significantly reduces vector storage and enables efficient approximate-distance calculations.
• Limitation: Quantization error can distort distances and change the nearest-neighbor ranking.
• Key settings: Number of subvectors and bits per code.
• Best fit: Collections where vector memory is a primary constraint.

Unlike HNSW or IVF, PQ is fundamentally a compression method. It can be used alone in a search index or combined with other indexing methods.

5. IVF-PQ — clustering plus compression

IVF-PQ combines two stages:

1. IVF: Choose a small number of promising vector clusters.
2. PQ: Score compact vector codes stored within those clusters.

• Strength: Reduces both the search scope and per-vector memory footprint.
• Limitation: Approximation comes from both unsearched clusters and quantized distances.
• Best fit: Very large RAG indexes where storing full embeddings in memory is too costly.

Some implementations also use residual PQ, compressing the difference between each vector and its assigned IVF centroid.

Practical RAG example

Suppose a knowledge base holds 10 million chunk embeddings.

• Flat: Compare the query against all 10 million vectors.
• HNSW: Navigate a graph to find promising neighbors.
• IVF: Search vectors in selected nearby clusters.
• PQ: Represent vectors as compact codes and estimate distances.
• IVF-PQ: Search selected clusters and score their compact codes.

After retrieval, a RAG pipeline may rerank candidate chunks before passing the most useful evidence to the LLM.

How should you choose?

Consider these priorities:

• Need an exact benchmark or have a small dataset? Start with Flat.
• Need fast interactive search with strong recall and can afford memory? Benchmark HNSW.
• Need cluster-pruned search with tunable query breadth? Consider IVF.
• Need to shrink vector storage? Consider PQ-based compression.
• Need both large-scale pruning and compression? Consider IVF-PQ.

There is no universally best index. Evaluate Recall@K, p95 retrieval latency, memory, build cost, and update behavior on representative queries.

Interview takeaway

Flat scans every vector and is exact; HNSW navigates a graph; IVF searches selected clusters; PQ compresses vectors into codes; and IVF-PQ combines IVF's cluster pruning with PQ's compression. The central trade-off is retrieval recall versus latency, memory, and operational cost.