What is an attention mask in an LLM?

An attention mask is a mechanism that tells an LLM which token positions are allowed to participate in attention and which positions should be ignored or blocked.

It is applied during the attention calculation so the model does not treat every token position as equally available.

Why is an attention mask needed?

Different tasks require different attention rules.

For example, an attention mask can be used to:

• Prevent a decoder-only model from seeing future tokens
• Ignore padding tokens in a batch
• Restrict attention to certain parts of a sequence
• Support specialized attention patterns

Conceptually:

allowed position → attention can use it
masked position  → attention is blocked

How does masking work?

Self-attention first produces compatibility scores between token positions.

Before those scores are converted into attention weights, the mask modifies the disallowed positions so that they receive effectively zero attention after softmax.

Conceptually:

attention scores
      ↓
apply mask
      ↓
blocked positions become unusable
      ↓
softmax
      ↓
attention weights

The exact implementation can vary, but the result is that masked positions do not contribute to the attention output.

What is a padding attention mask?

When sequences of different lengths are processed together, shorter sequences may contain padding tokens.

For example:

A: token token token token
B: token token <PAD> <PAD>

A padding mask tells the model not to treat those <PAD> positions as meaningful context.

What is a causal attention mask?

Decoder-only LLMs use a causal mask to prevent each position from attending to future positions.

For example:

Token 1 → can see Token 1
Token 2 → can see Tokens 1–2
Token 3 → can see Tokens 1–3

Future tokens are masked.

This preserves autoregressive next-token prediction.

The causal version is covered separately in “What is a causal attention mask?”

Is an attention mask itself a token?

No.

An attention mask is usually an additional tensor or set of rules used during the attention calculation.

It is different from special tokens such as <PAD> or <EOS>, although masks are often used together with those tokens.

Does every model use the same mask format?

No.

The exact representation depends on the architecture and framework.

Some systems use binary masks such as:

1 → allowed
0 → masked

while others internally convert masks into large negative values before softmax.

The principle is the same: control which positions can influence attention.

In short: an attention mask controls which token positions an LLM is allowed to attend to. It is commonly used to block future tokens, ignore padding, or enforce other attention patterns.