What is IVF-PQ?

IVF-PQ (Inverted File Index with Product Quantization) is an approximate nearest-neighbor search index that combines two techniques:

• IVF narrows the search to promising groups of vectors.
• PQ compresses the vectors in those groups into small codes.

Together, they make large vector collections more practical to search when storing every full-precision embedding is expensive.

How IVF-PQ is built

1. Learn IVF centroids: Cluster the embedding space into regions, commonly using k-means.
2. Assign vectors to inverted lists: Each vector belongs to the list associated with its nearest coarse centroid.
3. Compress the assigned vectors: PQ divides each vector representation into smaller subvectors and replaces them with learned codebook IDs. Many implementations apply PQ to the residual—the difference between a vector and its assigned IVF centroid—to improve approximation quality.
4. Store compact entries: An inverted list contains vector identifiers and their PQ codes instead of needing the entire full-precision vector for approximate search.

The index now has both a coarse map of where vectors are located and a compact way to represent them.

How a query is searched

For a RAG query:

1. Convert the user question into an embedding.
2. Find the nearest IVF centroids.
3. Select a limited number of their inverted lists, controlled by nprobe.
4. Score the compressed PQ codes in those lists using efficient approximate-distance calculations, often based on precomputed lookup tables.
5. Return the best Top-K candidates; optionally rerank them using original vectors or a separate reranking model if those resources are available.

Query → nearest IVF clusters → PQ-code distance estimates → Top-K chunks → optional reranking → LLM

Simple example

Imagine a RAG knowledge base with millions of document-chunk embeddings.

• An uncompressed flat index may need to compare the query with every full vector.
• IVF-PQ searches only selected clusters.
• Within those clusters, compact PQ codes let it evaluate candidates without reading each full-precision vector.

This can reduce both the search workload and the memory footprint.

What are the trade-offs?

IVF-PQ introduces two potential sources of recall loss:

• IVF pruning: A relevant vector may lie in a cluster that was not searched.
• PQ approximation: A compressed representation may slightly distort distances and change the ranking.

Important tuning parameters include:

• nlist: Number of IVF clusters created during indexing.
• nprobe: Number of clusters searched at query time.
• PQ subquantizer count and code size: Control compression strength and distance accuracy.

More searched clusters or less aggressive compression can improve recall, but may increase latency or memory usage.

When should you use IVF-PQ?

It is especially useful when the vector corpus is very large, memory is constrained, and some approximation is acceptable. It is not automatically better than HNSW or exact search: compare Recall@K, latency, index size, build time, and update behavior on the actual workload.

Interview takeaway

IVF-PQ combines IVF's cluster-based search pruning with PQ's compact vector representation. IVF decides where to search; PQ makes the vectors within those selected regions cheaper to store and compare. The result is memory-efficient ANN retrieval at scale, with recall affected by both cluster selection and quantization error.