Why combine semantic search with keyword search?

Combining semantic search with keyword search helps a RAG system retrieve evidence that matches both the *meaning* of a question and its *exact terms*. This combination is often called hybrid retrieval.

Why is one method not always enough?

• Semantic (dense) search uses embeddings to identify related ideas even when wording differs. It is useful for paraphrases but may overlook unusual identifiers, exact codes, or names.
• Keyword (lexical) search uses term matching, often ranked with BM25. It is strong at exact phrases and rare terms but can miss relevant passages that use synonyms.

Together, they cover different failure cases.

Practical example

Suppose a user asks: “Why does API PAY-409 fail when I submit a payment twice?”

• Keyword search can find a troubleshooting guide containing the precise code PAY-409.
• Semantic search can find another passage explaining *duplicate transaction requests*, even if it never uses the same words as the question.

Combining candidates gives the RAG assistant a better opportunity to retrieve the explanation and the exact reference.

How are the results combined?

1. Run keyword search and embedding-based search, often in parallel.
2. Retrieve a shortlist of document chunks from each.
3. Deduplicate chunks returned by both methods.
4. Fuse rankings, for example using Reciprocal Rank Fusion (RRF), which uses rank positions rather than assuming BM25 and vector similarity scores are comparable.
5. Optionally rerank the combined candidates against the full question.
6. Pass the strongest authorized, relevant chunks to the generator.

Question → lexical + semantic retrieval → fusion → reranking → answer

What improves in practice?

• Coverage: Better chance of finding answer-bearing chunks across different writing styles.
• Robustness: Less dependence on one retriever's weaknesses.
• Technical accuracy: Exact identifiers stay discoverable alongside conceptually related explanations.

Hybrid search is not guaranteed to be better for every query. It can require two indexes, more computation, and careful tuning. Validate the gain with labeled questions, Recall@K, p95 retrieval latency, and downstream grounded-answer quality.

Interview takeaway

Combine keyword search with semantic search because BM25-style retrieval captures exact terms while embeddings capture meaning and paraphrases. Fuse and deduplicate their results, then optionally rerank. The aim is more reliable evidence retrieval—not simply a larger set of chunks.