Dense Passage Retrieval for Open-Domain Question Answering

PAPER-DERIVED FACTS: DPR learns dense vector representations for questions and passages using two BERT encoders. Retrieval is performed with inner-product similarity and the resulting passages can be passed to a downstream reader.

PAPER-DERIVED FACTS: Traditional open-domain QA systems commonly relied on sparse retrieval such as BM25. DPR investigates whether a small amount of supervised QA data is enough to learn a semantic retriever that is both effective and computationally practical.

PAPER-DERIVED FACTS: The task is to retrieve passages containing answers from a large corpus given a natural-language question. The challenge is to learn semantic relevance while keeping retrieval fast enough for millions of passages.

PAPER-DERIVED FACTS: Sparse methods score lexical overlap, while dense retrieval maps queries and passages into continuous vector spaces. Bi-encoder architectures allow passage embeddings to be precomputed and indexed for efficient approximate nearest-neighbor search.

PAPER-DERIVED FACTS: DPR encodes a question q and passage p independently and scores them using the dot product qᵀp. Training uses positive passages and negative passages, including hard negatives mined from strong traditional retrievers.

PAPER-DERIVED FACTS: The retriever contains separate question and passage BERT encoders. Passage vectors are precomputed and stored in an index; at query time the question vector is compared against passage vectors and the top-k candidates are returned.

PAPER-DERIVED FACTS: Training uses open-domain QA datasets with question-answer pairs and supporting passages. Evaluation covers Natural Questions, TriviaQA, WebQuestions, CuratedTREC, and other open-domain retrieval/QA benchmarks.

PAPER-DERIVED FACTS: The dual encoders are optimized with a contrastive objective that increases the score of positive passages relative to negatives. Hard-negative passages are important because random negatives are often too easy.

PAPER-DERIVED FACTS: The study evaluates retrieval recall and end-to-end QA after passing retrieved passages to a reader. It compares DPR against BM25 and earlier retrieval systems across multiple QA datasets.

PAPER-DERIVED FACTS: The principal baseline is Lucene BM25, along with prior learned and hybrid retrieval systems. End-to-end comparisons use readers operating on retrieved candidate passages.

PAPER-DERIVED FACTS: DPR improves top-20 passage retrieval accuracy over a strong BM25 baseline by large margins on the reported datasets, and the resulting reader systems achieve new state-of-the-art results on several open-domain QA benchmarks.

PAPER-DERIVED FACTS: The paper studies negative sampling, hard negatives, training-data size, and encoder configurations. The results show that supervised semantic learning and informative negatives are key to dense retrieval quality.

PAPER-DERIVED FACTS: Dense retrieval can miss passages whose relevance depends on exact lexical cues or rare entities, and retrieval quality remains sensitive to the training distribution. Errors at the retriever stage propagate to the downstream reader.

PAPER-DERIVED FACTS: DPR demonstrates that simple dense dual encoders can provide strong open-domain passage retrieval at scale. Its design became a core building block for subsequent RAG architectures.

PAPER-DERIVED FACTS: Primary source: Karpukhin, V. et al. (2020), Dense Passage Retrieval for Open-Domain Question Answering. EMNLP 2020. arXiv:2004.04906.