How does RAG work end to end?

RAG works in two major phases: prepare the knowledge first, then retrieve the right pieces when a user asks a question.

1. Ingestion: prepare the knowledge

Suppose your source is an employee handbook.

The ingestion flow is:

Documents
   ↓
Parse text / tables / metadata
   ↓
Split into chunks
   ↓
Create embeddings
   ↓
Store chunks + vectors + metadata

For example, a 100-page handbook might become hundreds of searchable chunks.

2. User asks a question

Example:

What is the notice period for senior engineers?

The system does not immediately send this question to the LLM.

First, it tries to find the relevant evidence.

3. Convert the query for retrieval

For vector retrieval, the query is converted into an embedding.

User query
   ↓
Embedding model
   ↓
Query vector

If the system also uses keyword search such as BM25, the original query can be sent to that retriever as well.

4. Retrieve candidate chunks

The retriever searches the knowledge base and returns the most relevant chunks.

For example:

Chunk 1: Senior engineers have a 60-day notice period.
Chunk 2: Junior engineers have a 30-day notice period.

Retrieval can use:

• Dense vector search
• Sparse or keyword search
• Hybrid retrieval
• Metadata filters

5. Optional reranking

The first retriever is optimized to find good candidates quickly, not necessarily to perfectly order them.

A reranker can examine the query and retrieved chunks more carefully and move the strongest evidence to the top.

Retrieve 20 candidates
        ↓
Rerank
        ↓
Keep best 5

6. Build the LLM context

The selected chunks are placed into the prompt together with the user's question and instructions.

Conceptually:

System instruction
+ retrieved evidence
+ user question

A common instruction is to answer from the supplied evidence and say when the answer is not available.

7. Generate the answer

The LLM now receives both the question and relevant external knowledge.

For example:

Retrieved evidence:
Senior engineers have a 60-day notice period.

Question:
What is the notice period for senior engineers?

Answer:
The notice period is 60 days.

8. Return citations or sources

A good RAG system often keeps source metadata such as document name, page number, URL, or section so the final answer can point back to the evidence.

End-to-end picture

INGESTION
Documents → Parse → Chunk → Embed → Index

RETRIEVAL
Question → Retrieve → Rerank → Select context

GENERATION
Question + Context → LLM → Grounded answer

Interview takeaway

RAG first converts external knowledge into searchable units. At query time it retrieves the most relevant evidence, optionally reranks it, places that evidence into the prompt, and lets the LLM generate an answer grounded in the retrieved context.