What is keyword search in RAG?

Keyword search in RAG retrieves document chunks based on the words or terms appearing in a user's question. It is a lexical retrieval technique: rather than converting every query into a semantic vector, it looks for matching indexed terms and ranks the matching chunks.

How does it work?

1. Index the documents: Parse source material into chunks and build a searchable index of their words, typically an inverted index mapping terms to chunk IDs.
2. Process the question: Analyze the query text, often including tokenization and optional lowercasing, stemming, or synonym handling.
3. Find matching chunks: Look up chunks containing query terms or supported phrase matches.
4. Rank the matches: Use a lexical scoring method such as BM25 to favor informative matches, taking term rarity, frequency, and chunk length into account.
5. Pass evidence to RAG: Return the strongest matching chunks, optionally combine or rerank them, and supply the resulting context to the language model.

Example

A user asks: “What causes payment error PAY-409?”

The support guide includes a section titled “PAY-409: Duplicate transaction.”

Keyword search can retrieve that section by matching the precise error identifier. This is particularly valuable because a semantic embedding may not reliably distinguish similar-looking error codes.

When is keyword search useful?

• Exact identifiers: Error codes, invoice IDs, product SKUs, API endpoint names, or configuration keys.
• Rare technical terminology: Terms that must match closely.
• Named entities: Specific product names, legal clauses, or feature labels.
• Auditable retrieval: It can be easier to explain which search terms caused a chunk to rank highly.

Keyword search versus semantic search

• Keyword search: Uses lexical matches and works especially well when the precise wording matters.
• Semantic search: Uses embeddings to retrieve conceptually similar text even when the words differ.
• Hybrid retrieval: Combines lexical and semantic candidate sets to benefit from both approaches.

Keyword search can miss relevant passages that use synonyms or paraphrases. For example, a query about “work from home” may not strongly match a document that only says “remote employment.” Semantic retrieval can help bridge that wording gap.

Is keyword search the same as BM25?

Not exactly. Keyword search is the broader retrieval approach; BM25 is one ranking algorithm used within it. Other keyword search implementations may use Boolean matching, phrase searches, or other lexical scoring methods.

Practical tips for RAG

• Preserve codes, punctuation-sensitive identifiers, and important product terms during text processing.
• Tune analyzers and field boosts to suit the document types.
• Apply metadata and authorized-access restrictions before returning chunks.
• Measure whether answer-bearing evidence appears in Top-K, especially for identifier-heavy queries.

Interview takeaway

Keyword search retrieves RAG evidence by matching words or phrases in an indexed document collection. It is reliable for exact terms and identifiers, while semantic search handles paraphrases. In practice, many RAG systems combine both using hybrid retrieval.