What are query, key, and value representations in LLM attention?

In LLM attention, query, key, and value representations are three learned versions of each token representation that help the model decide which other tokens are relevant and what information to take from them.

They are commonly abbreviated as:

• Q = Query
• K = Key
• V = Value

What does the query represent?

The query represents what the current token is looking for.

For example, if the model is processing a pronoun such as:

she

its query can help identify which earlier token is most relevant to that pronoun.

What does the key represent?

The key represents information used to determine how relevant a token is to another token's query.

The model compares a query with the keys of allowed tokens.

A stronger match generally produces a higher attention score.

Conceptually:

Query of current token
        ↓
compared with
        ↓
Keys of other tokens
        ↓
attention scores

What does the value represent?

The value contains the information that can actually be passed forward if that token receives attention.

After the model calculates attention weights from the query-key comparisons, it uses those weights to combine the corresponding value representations.

So:

Query + Keys
     ↓
attention weights
     ↓
weighted Values
     ↓
context-aware representation

Where do Q, K, and V come from?

Each token starts with an internal representation.

The model applies learned linear transformations to create separate query, key, and value vectors:

token representation
   ├──→ Query
   ├──→ Key
   └──→ Value

These transformations are learned during training.

Why use three different representations?

Using separate Q, K, and V representations gives the model different roles for:

• Asking what information is relevant
• Describing what information a token can match against
• Carrying the information that will be combined

This makes attention flexible because the same token can be useful in different ways depending on the surrounding context.

Simple example

Consider:

The developer fixed the bug because she understood the issue.

When processing:

she

its query may strongly match the key associated with:

developer

That token can therefore receive a higher attention weight, allowing its value representation to contribute more strongly to the representation of she.

Are Q, K, and V the same across all attention heads?

No.

In standard multi-head attention, different heads usually learn their own query, key, and value projections.

This allows different heads to focus on different relationships in the same sequence.

That idea is covered separately in “What is multi-head attention in an LLM?”

In short: queries determine what a token is looking for, keys help measure which tokens are relevant, and values carry the information that is combined according to the resulting attention weights.