How should code be chunked for RAG?

Code should usually be chunked for RAG using logical program structure rather than only fixed character or token counts.

Source code has natural boundaries such as files, classes, functions, methods, and modules. Preserving those boundaries makes each retrieved chunk easier to understand and more useful for code-related questions.

Prefer semantic code units

Useful chunk boundaries include:

• A complete function
• A complete method
• A class
• A small module
• A configuration block
• A related group of constants or definitions

For example, instead of splitting this code every 500 tokens, keep the function intact:

PYTHON
def calculate_total(items):
    total = 0
    for item in items:
        total += item.price
    return total

A complete function is usually a better retrieval unit than a fragment containing only half of its logic.

Preserve the structural context

A code chunk should retain enough surrounding information to explain where it belongs.

Useful metadata can include:

• Repository name
• File path
• Programming language
• Class name
• Function or method name
• Module or package name
• Start and end line numbers

For example:

repository: checkout-service
file: src/payments/refund.py
class: RefundService
method: create_refund
lines: 82-131

This metadata can improve filtering, navigation, and source attribution during retrieval.

Keep related comments and documentation with the code

Docstrings and comments often explain:

• What a function does
• Expected inputs
• Return values
• Constraints
• Edge cases

When possible, keep them with the code they describe.

For example:

PYTHON
def validate_token(token):
    """Validate the access token and return its claims."""
    ...

Separating the docstring from the function can remove useful semantic context.

Handle large functions or classes carefully

Some code units may be too large for a practical retrieval chunk.

In that case, split them at smaller structural boundaries such as:

• Methods inside a class
• Logical blocks inside a very large function
• Groups of closely related helper functions

The parent class or function identity should still be preserved in metadata.

Preserve important dependencies

A code chunk may depend on symbols defined elsewhere.

Useful context can include:

• Imported modules
• Called functions
• Referenced classes
• Interfaces
• Parent classes

You do not necessarily need to duplicate all dependency code inside every chunk, but preserving symbol names and relationships helps retrieval systems connect related code.

Avoid arbitrary mid-statement splits

Blind fixed-size chunking can produce fragments such as:

Chunk 1 → first half of a function
Chunk 2 → second half of the function

These fragments may be syntactically incomplete and harder to interpret.

If a size limit must be used, first split using code-aware boundaries and then apply token limits within those units.

Consider language-aware parsing

Code parsers, abstract syntax trees, or language-specific splitters can identify structural elements more reliably than plain-text delimiters.

For example, a parser can detect:

• Functions in Python
• Methods and classes in Java
• Components and functions in JavaScript or TypeScript
• Structs and functions in Go

This is especially useful for large repositories containing many files and languages.

Add relationships when repository-level questions matter

For questions such as:

Where is this function called?

or:

Which service uses this class?

chunk text alone may not be enough.

The ingestion pipeline can additionally preserve relationships such as:

• Function calls
• Imports
• Class inheritance
• File dependencies

These relationships can later support graph-based or metadata-assisted retrieval.

A practical code-chunking flow

Parse repository
      ↓
Identify files and language
      ↓
Extract classes/functions/methods
      ↓
Keep comments and docstrings
      ↓
Split oversized code units if necessary
      ↓
Attach file and symbol metadata
      ↓
Embed and index chunks

In short, code should be chunked around meaningful program structures such as functions, methods, and classes, while preserving file paths, symbol names, comments, and dependency context. Structure-aware code chunks are generally more useful for RAG than arbitrary fixed-size fragments.