What is an encoder-decoder architecture?

An encoder-decoder architecture is a neural network design with two main parts: an encoder that converts the input into an internal representation and a decoder that uses that representation to produce the output.

It is commonly used when the input and output have different structures or lengths.

What does the encoder do?

The encoder reads the input and transforms it into a representation that captures the most useful information.

For example:

Input sequence → Encoder → Internal representation

In older sequence models, this representation may be a fixed hidden state. In modern attention-based architectures, the decoder can often access richer encoder representations.

What does the decoder do?

The decoder takes the encoded information and generates the required output.

For sequence generation, it may produce one element at a time:

Internal representation → Decoder → Output sequence

Simple example

In machine translation:

English sentence → Encoder → Representation → Decoder → French sentence

The encoder focuses on understanding the input, while the decoder focuses on generating the target sequence.

Where is encoder-decoder architecture used?

Common applications include:

• Machine translation
• Text summarization
• Speech recognition
• Image captioning
• Sequence-to-sequence learning

What models use this architecture?

Encoder-decoder designs can be built with:

• RNNs
• LSTMs
• GRUs
• Transformers

Attention mechanisms are often added so the decoder can focus on the most relevant encoder information while generating each output.

For the sequence-learning use case, see “What is sequence-to-sequence learning?”. For the attention component, see “What is the attention mechanism?”.

In short, an encoder-decoder architecture separates a model into one component that represents the input and another that generates the desired output from that representation.