How does IVF-PQ reduce memory usage?

IVF-PQ reduces memory usage primarily by replacing full-precision embedding vectors with compact Product Quantization (PQ) codes. IVF organizes those compressed vectors into searchable clusters; it is PQ, not clustering alone, that provides most of the per-vector memory savings.

How compression reduces storage

A standard embedding often stores every dimension as a 32-bit floating-point value.

For example, a 768-dimensional vector needs:

• 768 × 4 bytes = 3,072 bytes (about 3 KB) per vector.
• 10 million vectors ≈ 30.72 GB for the raw embeddings alone, excluding index and database overhead.

Now consider a common illustrative PQ configuration:

• Split each 768-dimensional embedding into 96 subvectors, each containing 8 dimensions.
• Learn a codebook of 256 representative centroids for each subvector position.
• Store an 8-bit centroid ID for each subvector instead of its original eight floats.

The compressed representation contains 96 bytes per vector rather than 3,072 bytes.

That is 32× smaller for the vector representation. Across 10 million vectors, the codes alone require roughly 0.96 GB, rather than 30.72 GB.

These figures are illustrative—not a promise of the same total index-size reduction in every implementation.

What IVF adds

IVF assigns vectors to inverted lists based on nearby coarse centroids. In many IVF-PQ implementations, PQ encodes the residual vector (the difference between a vector and its assigned centroid), which can improve compression accuracy.

The index stores compact PQ codes and identifiers inside those lists. At search time:

Query embedding → choose nearby IVF lists → estimate distances using PQ codes → return the closest candidates

This avoids keeping and scanning every original full-precision vector in the active search index.

What still consumes memory?

The full IVF-PQ index is larger than its PQ codes alone because it may also hold:

• Vector identifiers and inverted-list bookkeeping.
• IVF centroids and PQ codebooks.
• Lookup tables or temporary search structures.
• Additional metadata, filtering indexes, and database overhead.
• Optionally, original vectors stored separately for reconstruction or accurate reranking.

For example, an 8-byte vector ID would add another 8 bytes per entry in an implementation using that ID width. The exact total depends on the library and configuration.

What is the trade-off?

PQ is lossy: the stored codes approximate the original vectors. This introduces quantization error, which can change similarity rankings and reduce Recall@K.

IVF introduces another source of approximation when a query searches only some inverted lists. You therefore tune both:

• PQ code length: More bits generally preserve more information but use more memory.
• nprobe: Searching more IVF lists generally improves recall but increases latency.

If recall is critical, some systems retrieve a larger candidate pool with IVF-PQ and rerank it using full-precision vectors or another ranking method.

Interview takeaway

IVF-PQ saves memory because it stores short PQ codes instead of full floating-point embeddings. IVF narrows which clusters to search; PQ compresses the vectors in those clusters. A 768-dimensional FP32 vector (3,072 bytes) can, for example, become a 96-byte PQ code—32× smaller for the vector representation—while index metadata and optional original vectors add overhead. The main cost is approximate distances and possible retrieval-recall loss.