How is a user query converted into an embedding?

A user query is converted into an embedding by passing the query text through the same or a compatible embedding model used for the indexed document chunks.

This happens at retrieval time, after the user submits a question and before vector similarity search begins.

1. Start with the user's query

For example:

How long can I request a refund?

The system first takes the query text exactly as the retrieval pipeline expects to process it.

2. Apply any query preprocessing

Depending on the RAG system, the query may be lightly prepared before embedding.

This can include:

• Trimming unnecessary whitespace
• Normalizing text
• Removing unsupported characters
• Adding a task-specific prefix required by the embedding model

Some embedding models are designed to use different prefixes or instructions for queries and documents.

For example, conceptually:

query: How long can I request a refund?

The exact formatting depends on the embedding model.

3. Tokenize the query

The query text is converted into tokens by the embedding model's tokenizer.

Conceptually:

Query text
    ↓
Tokenizer
    ↓
Token IDs

Because user queries are usually much shorter than document chunks, they normally fit comfortably within the model's input limit.

4. Pass the tokens through the embedding model

The tokenized query is processed by the embedding model.

Tokenized query
      ↓
Embedding model
      ↓
Query vector

The result is a fixed-length numerical vector representing the semantic meaning of the query.

For example:

[0.084, -0.215, 0.602, 0.117, ...]

The real vector may contain hundreds or thousands of dimensions depending on the model.

5. Compare the query vector with stored document vectors

The query embedding is not normally stored permanently just to answer one request.

Instead, it is used immediately to search the vector index.

A simplified flow is:

User query
   ↓
Query embedding
   ↓
Compare with stored chunk embeddings
   ↓
Rank by vector similarity
   ↓
Return the most relevant chunks

Similarity may be calculated using methods such as cosine similarity, dot product, or another metric supported by the embedding model and vector index.

6. Retrieve the closest document chunks

The vector database returns chunks whose embeddings are closest to the query embedding.

For example, the query:

How long can I request a refund?

may retrieve:

The refund window is 30 days from the date of purchase.

even though the wording is different.

That semantic matching is the main reason the query is embedded.

The query and documents must share a compatible vector space

The query embedding must be comparable with the stored document embeddings.

This generally means using the same embedding model or a model pair explicitly designed to produce compatible query and document representations.

If unrelated models are used, the vectors usually do not share a meaningful coordinate space.

This is covered in “Why should document and query embeddings use compatible embedding models?”.

Query embeddings are created at runtime

Document embeddings are usually generated during ingestion and stored in advance.

Query embeddings are generated when the user asks a question.

Document chunks → embedded during ingestion
User query      → embedded during retrieval

This lets the system compare each new question against the precomputed document vectors.

Query preprocessing should be consistent

Any preprocessing required by the embedding model should be applied consistently.

For example, if the model expects a special query instruction, omitting it can reduce retrieval quality.

The retrieval pipeline should follow the embedding model's recommended input format for both query and document representations.

In short, a user query is converted into an embedding by preparing and tokenizing the query, passing it through a compatible embedding model, and obtaining a fixed-length query vector. That vector is then compared with stored document-chunk embeddings to retrieve semantically relevant content.