If you are using BM25 + vector search, how do you combine the results?

When you use BM25 and vector search together, you usually run both retrievers, collect their candidate results, then fuse the two ranked lists into one final ranking.

The important point is that BM25 scores and vector-similarity scores are usually on different scales, so you should not blindly add the raw numbers.

Step 1: Run both retrievers

Example query:

"How do I fix ERR_AUTH_401 after my token expires?"

BM25 might return:

1. Chunk B
2. Chunk A
3. Chunk E

Vector search might return:

1. Chunk A
2. Chunk C
3. Chunk D

BM25 likes Chunk B because it contains the exact error code.

Vector search likes Chunk A because it matches the meaning of the query.

Step 2: Merge and deduplicate

Take the union of candidates from both retrievers.

For example:

A, B, C, D, E

If the same chunk appears in both lists, keep one copy but remember that both retrievers found it.

That overlap is often a strong relevance signal.

Step 3: Combine the rankings

There are several common approaches.

Reciprocal Rank Fusion

RRF combines rank positions rather than raw scores.

A document ranked highly by either retriever gets a useful contribution, and a document ranked highly by both usually moves further up.

This is popular because BM25 and vector scores do not need to be directly comparable.

Weighted score fusion

Another option is to normalize the scores from each retriever and combine them.

Conceptually:

Final score = 0.4 × BM25 score + 0.6 × vector score

The weights depend on your workload.

If exact identifiers matter heavily, you may give BM25 more influence.

If semantic questions dominate, vector search may get more weight.

The key requirement is to normalize the scores first if their scales differ.

Step 4: Optionally rerank

After fusion, you may have a candidate list such as:

Top 20 fused results → reranker → best 5

A cross-encoder or LLM-based reranker can inspect the query and each candidate more deeply than the initial retrievers.

This is often where the final ordering improves significantly.

Example

Suppose Chunk A is:

"Authentication fails when an access token has expired."

Chunk B is:

"ERR_AUTH_401: access token expired."

Vector search may rank A first.

BM25 may rank B first.

Fusion keeps both strong candidates instead of forcing you to choose only one retrieval method.

Which fusion method should you start with?

A practical starting point is:

BM25 + vector search → RRF → optional reranker

RRF is attractive because it is simple and avoids the problem of incompatible raw score scales.

Weighted score fusion can work very well too, but it requires more calibration.

Interview takeaway

When combining BM25 and vector search, retrieve candidates from both, merge and deduplicate them, then fuse their rankings using a method such as Reciprocal Rank Fusion or normalized weighted scores. After fusion, a reranker can reorder the strongest candidates before the final context is sent to the LLM.