What is a stop sequence in LLM generation?

A stop sequence is a predefined text pattern that tells an LLM serving system to stop generating when that pattern appears in the newly generated output.

It acts as a stopping rule during inference—not as a change to the model's learned weights.

Simple example

Imagine an application asks the model to answer a question and uses this stop sequence:

Stop sequence: ### User:

The model starts generating:

The answer is 42.
### User:

When the system detects ### User:, it stops generation. Many APIs omit the matched stop sequence from the response returned to the application.

How does it work?

1. The application configures one or more stop sequences.
2. The LLM generates tokens as usual.
3. The serving system checks the generated text for a matching sequence.
4. When a match occurs, generation stops instead of continuing.

A stop sequence can contain multiple characters and may span multiple tokens; it does not have to match a single tokenizer token.

Why use stop sequences?

• Control response boundaries: Prevent a model from continuing into another dialogue turn or section.
• Separate structured sections: Stop before a known delimiter in generated text.
• Avoid unnecessary output: End generation once a particular marker appears.

What if the model never generates the sequence?

Then that stop rule does not trigger. Generation may instead end because of a model end-of-sequence token, an output-token limit, or another application stop condition.

Stop sequence vs. end-of-sequence token

• Stop sequence: Usually a custom text pattern configured by the application or inference API.
• End-of-sequence (EOS) token: A special token used by the model/tokenizer to signal a sequence ending.
• Maximum output tokens: A length limit that stops generation even when no delimiter or EOS appears.

Common mistake

Avoid choosing a delimiter that appears frequently in ordinary answers. For example, using a period (.) as a stop sequence could cut off a response unexpectedly. Prefer a distinctive marker.

Exact matching and whether the marker is included in the returned text depend on the API or inference engine.

In short: a stop sequence is an application-defined output delimiter that terminates LLM generation when detected, helping control where a response ends.