How does nDCG reward highly relevant documents appearing earlier?

Normalized Discounted Cumulative Gain (nDCG) rewards retrieval systems that place more relevant documents near the top of their ranked results. It uses two ideas: highly relevant results contribute more gain, and results appearing later receive a position discount.

Why does ranking position matter?

In RAG, the generator often receives only the top few retrieved chunks. A highly relevant passage at rank 1 is usually more useful than the same passage buried at rank 20, where it may never enter the context window.

Unlike binary metrics that label passages only “relevant” or “not relevant,” nDCG can use graded relevance, such as:

• 0: Not relevant
• 1: Somewhat relevant
• 2: Relevant
• 3: Highly relevant

Formula

For the top K results, a common definition is:

DCG@K = Σᵢ₌₁ᴷ (2^relᵢ − 1) / log₂(i + 1)

Where:

• relᵢ is the relevance grade of the document at rank i.
• 2^relᵢ − 1 rewards higher relevance grades disproportionately.
• log₂(i + 1) discounts documents that appear farther down.

nDCG@K = DCG@K / IDCG@K

IDCG@K is the highest possible DCG@K obtained by ordering the available relevance grades ideally. nDCG is typically between 0 and 1, where 1 indicates ideal ranking. Queries with no relevant documents need an explicit evaluation convention.

Beginner-friendly numerical example

Three retrieved documents have relevance grades 3, 2, 0.

Ranking A: [3, 2, 0]

• Rank 1 gain: (2³ − 1)/log₂(2) = 7
• Rank 2 gain: (2² − 1)/log₂(3) ≈ 1.89
• Rank 3 gain: 0
• DCG@3 ≈ 8.89; this is the ideal ranking, so nDCG@3 = 1.00.

Ranking B: [0, 2, 3]

• Rank 1 gain: 0
• Rank 2 gain: 1.89
• Rank 3 gain: (2³ − 1)/log₂(4) = 3.50
• DCG@3 ≈ 5.39
• nDCG@3 ≈ 5.39/8.89 ≈ 0.61.

Both rankings retrieve the same documents, but Ranking A scores higher because the strongest evidence appears first.

When is nDCG useful for RAG?

• Comparing embedding models, rerankers, or hybrid-search configurations.
• Evaluating results where some passages directly answer the question and others provide only background.
• Measuring improvements in the order of retrieved evidence—not merely whether evidence appeared.

Important limitations

• nDCG depends on consistent relevance labels and a clearly defined K.
• High nDCG does not guarantee that the generated answer is correct or faithful.
• It does not directly measure whether all evidence required for multi-hop questions was retrieved.

Interview takeaway

nDCG rewards both relevance strength and ranking position. Placing highly relevant passages earlier produces a higher score even when the set of retrieved documents stays unchanged.