What is a causal attention mask?

A causal attention mask is a mask used in autoregressive language models to prevent each token from attending to tokens that come later in the sequence.

It enforces the rule:

current token → can use itself and earlier tokens
current token → cannot use future tokens

Why is it needed?

During training, the complete sequence is available to the model.

Without a causal mask, a token could directly access future tokens, including information it is supposed to predict.

That would break next-token prediction.

The causal mask ensures the model learns under the same information constraint used during generation.

What does the mask look like?

For a sequence of four tokens, the allowed attention pattern is conceptually:

Token 1 → 1
Token 2 → 1, 2
Token 3 → 1, 2, 3
Token 4 → 1, 2, 3, 4

Positions to the right are blocked.

This creates a triangular attention pattern.

How is the mask applied?

Self-attention first computes scores between token positions.

The causal mask then marks future positions as disallowed before softmax is applied.

Conceptually:

attention scores
      ↓
apply causal mask
      ↓
future positions blocked
      ↓
softmax
      ↓
valid attention weights

In many implementations, blocked positions are assigned a very large negative value so their probability becomes effectively zero after softmax.

How does this support text generation?

Suppose the sequence so far is:

The weather is

When predicting the next token, the model can use:

The
weather
is

but it cannot use tokens that have not yet been generated.

After a new token is produced, it is added to the sequence and becomes available for the next prediction.

Is a causal mask the same as a padding mask?

No.

A causal mask blocks future positions.

A padding mask blocks padding tokens that should not contribute useful information.

A model may use both kinds of masking at the same time.

Is a causal attention mask the same as causal self-attention?

They are closely related but not identical concepts.

• Causal attention mask is the masking rule that blocks future positions.
• Causal self-attention is self-attention operating under that causal restriction.

For the broader mechanism, see “What is causal self-attention?”

In short: a causal attention mask prevents tokens from seeing future positions, creating the left-to-right information flow required for autoregressive next-token prediction.