What is OCR and when is it needed in RAG?

OCR (Optical Character Recognition) is the process of converting text that appears inside an image into machine-readable text.

In RAG, OCR is needed when important document content cannot be extracted directly as normal digital text.

When OCR is needed

OCR is commonly required for:

• Scanned PDFs
• Photos of documents
• Faxed documents
• Screenshots containing text
• Image-only pages
• Labels or text embedded inside diagrams

For example, a scanned contract may look like a normal PDF to a person, but internally each page may only be an image. Without OCR, the RAG pipeline may have little or no text to index.

When OCR is not needed

If a PDF or document already contains a reliable text layer, native text extraction is usually preferable.

Using OCR unnecessarily can:

• Add processing cost
• Introduce recognition errors
• Lose formatting or structure
• Misread numbers, symbols, or technical terms

A good ingestion pipeline therefore detects whether usable text already exists before applying OCR.

What OCR produces

OCR can extract visible text such as:

• Words and sentences
• Headings
• Page numbers
• Labels
• Table cell text

Some OCR systems can also return useful information such as text coordinates, blocks, and confidence scores.

OCR does not fully understand layout

Recognizing characters is only part of the problem.

A page may contain:

• Multiple columns
• Tables
• Forms
• Charts
• Diagrams
• Sidebars

Even when OCR reads the words correctly, the RAG pipeline may still need layout analysis to determine the proper reading order and relationships between elements.

OCR quality affects retrieval quality

OCR errors become part of the indexed knowledge if they are not detected.

Common errors include:

• 0 confused with O
• 1 confused with I
• Missing punctuation
• Broken words
• Incorrect reading order
• Misread technical identifiers

These errors can reduce retrieval accuracy because the indexed text no longer matches the original information.

Validate OCR before indexing

Before chunking OCR output, check representative pages for:

• Missing text
• Incorrect characters
• Reading-order problems
• Lost headings
• Broken tables
• Important numeric errors

When OCR confidence is available, very low-confidence regions can be flagged for reprocessing or review.

Document page
     ↓
Usable native text?
  ↙             ↘
Yes              No
 ↓                ↓
Extract text      OCR
       \          /
        Validate
           ↓
      Chunk + index

For the full scanned-document workflow, see “How should scanned PDFs be handled in RAG?”.

In short, OCR is used in RAG when useful text exists only inside images or scanned pages. It converts that visible text into machine-readable form, but the result should still be validated and combined with layout and source metadata before indexing.