What is a variational autoencoder (VAE)?

A Variational Autoencoder (VAE) is a generative neural network that learns a structured latent space and can generate new data by sampling from that space.

It is related to a standard autoencoder, but instead of mapping an input to one fixed latent vector, a VAE learns a probability distribution for the latent representation.

How a VAE works

A VAE has two main parts:

• Encoder: converts the input into parameters of a latent probability distribution, commonly a mean and variance.
• Decoder: reconstructs or generates data from a sampled latent vector.

A simplified flow is:

Input → Encoder → Mean and variance → Sample latent vector → Decoder → Reconstruction

Why does the encoder output a distribution?

Learning a distribution makes the latent space smoother and more organized.

Nearby points in that latent space are encouraged to represent similar kinds of data, which makes it possible to sample new latent vectors and generate new examples.

What loss does a VAE use?

A VAE usually combines two objectives:

• Reconstruction loss: encourages the decoded output to resemble the original input.
• KL divergence: encourages the learned latent distribution to stay close to a chosen prior distribution, commonly a standard normal distribution.

A simplified objective is:

VAE loss = reconstruction loss + KL divergence

What is the reparameterization trick?

Sampling directly from a learned distribution would normally make gradient-based training difficult.

VAEs use the reparameterization trick so sampling can be expressed in a way that still allows gradients to flow through the encoder during backpropagation.

What are VAEs used for?

Common uses include:

• Generating images or other data
• Learning compact latent representations
• Data interpolation
• Anomaly detection
• Representation learning

For the simpler non-generative version, see “What is an autoencoder?”. For the broader category, see “What is a generative model in deep learning?”.

In short, a VAE learns a probability-based latent space, then samples from that space to reconstruct existing data or generate new examples.