What is a repetition penalty in LLM generation?

A repetition penalty is a decoding control that discourages an LLM from selecting tokens it has already used, helping reduce repetitive words, phrases, and loops during text generation.

It adjusts the next-token scores during inference; it does not retrain the model or change its weights.

Why is it needed?

An LLM generates text one token at a time. Sometimes it repeatedly favors the same continuation:

The result is good. The result is good. The result is good.

A repetition penalty makes previously seen tokens less attractive, increasing the chance of a different continuation.

How does it work?

In a common implementation, the decoder:

1. Identifies tokens appearing in a configured history (often the prompt and generated output, depending on the system).
2. Adjusts their logits (next-token scores) to discourage reusing them.
3. Selects the next token using the chosen decoding strategy, such as greedy decoding or sampling.
4. Repeats this process at each generation step.

For a common multiplicative repetition-penalty convention:

• 1.0: No repetition penalty.
• Greater than 1.0: Discourages previously seen tokens more strongly.
• Less than 1.0: Can favor repetition, where supported.

Exact parameter behavior varies between inference engines and APIs.

What are the trade-offs?

A modest penalty may improve diversity and reduce loops. Too much penalty can damage fluency because some repetition is necessary for coherent writing.

For example, an explanation may need to repeat a technical term, a person's name, or a key variable. An excessive penalty may encourage awkward substitutions instead.

Is it the same as frequency or presence penalties?

Not necessarily.

• A repetition penalty typically modifies scores for tokens that have appeared before.
• A presence penalty commonly applies a penalty once a token has appeared, regardless of its count.
• A frequency penalty commonly increases the penalty with the number of prior occurrences.

These are different mechanisms, and providers do not always implement or name them identically.

Does it guarantee no repetition?

No. It changes token-selection preferences rather than imposing an absolute ban. A model may still repeat content when that continuation remains likely or necessary.

For background, see “What are decoding strategies in LLMs?”

In short: a repetition penalty discourages reuse of previously seen tokens during decoding. Used moderately, it can reduce repetitive output; set too high, it can make responses unnatural or less coherent.