How do the update and reset gates work in a GRU?

A GRU (Gated Recurrent Unit) uses two learned gates to control how its hidden state changes as a sequence is processed. The reset gate decides how much previous information contributes to a *candidate* hidden state. The update gate decides how much of the *old* hidden state to retain versus replace with that candidate.

What does each gate do?

At step t, the GRU receives the current input xₜ and previous hidden state hₜ₋₁. Both gates use a sigmoid activation, giving continuous values between 0 and 1 for each hidden-state component.

• Reset gate (`rₜ`): A value near 0 makes the proposed new content depend less on the previous hidden state; near 1 lets the candidate use more of that history. Resetting the candidate's access to history does not directly erase stored memory.
• Update gate (`zₜ`): Using the convention below, a value near 1 mostly retains the previous hidden state; near 0 mostly takes the new candidate. One gate controls both retention and replacement.

GRU equations

rₜ = σ(Wᵣxₜ + Uᵣhₜ₋₁ + bᵣ) — reset gate

zₜ = σ(W𝓏xₜ + U𝓏hₜ₋₁ + b𝓏) — update gate

nₜ = tanh(Wₙxₜ + Uₙ(rₜ ⊙ hₜ₋₁) + bₙ) — candidate hidden state

hₜ = zₜ ⊙ hₜ₋₁ + (1 − zₜ) ⊙ nₜ — updated hidden state

Here σ is sigmoid; tanh squashes the candidate into −1 to 1; W, U, and b are trainable weights and biases; and ⊙ means element-wise multiplication. All equations act on vectors in a real GRU, but one scalar component is enough for a numerical example.

The reset gate acts first, while forming the candidate; the update gate acts second, mixing candidate and prior state.

Beginner-friendly numerical example

Suppose one hidden-state component is `hₜ₋₁ = 0.8`. For illustration, learned gate calculations yield pre-sigmoid scores of −1.0986 for reset and +1.0986 for update. Assume the candidate's current-input contribution is 0.4, its recurrent weight is 0.5, and its bias is zero.

1. Reset gate: rₜ = σ(−1.0986) ≈ 0.25.
2. Update gate: zₜ = σ(+1.0986) ≈ 0.75.
3. Candidate: The reset-scaled prior state is 0.25 × 0.8 = 0.2. Therefore, nₜ = tanh(0.4 + 0.5 × 0.2) = tanh(0.5) ≈ 0.4621.
4. Final state: hₜ = 0.75 × 0.8 + (1 − 0.75) × 0.4621 = 0.6 + 0.1155 ≈ 0.7155.

Interpretation: The reset gate allowed only 25% of the old hidden-state component into the *candidate calculation*. The update gate then retained 75% of the old hidden state in the *final result*, mixing in 25% candidate. So the model can propose new content using little history while still retaining most of its previous representation. All sample values are illustrative, not measured from a trained model.

How is a GRU different from an LSTM?

A GRU has one recurrent hidden state, which it uses for both memory and output, rather than the separate hidden state and cell state of an LSTM. Its reset and update gates provide a compact mechanism for selective candidate formation and retention. For the dedicated LSTM details, see “How do the input, forget, and output gates work in an LSTM?” (Question 243) and “What is the difference between an LSTM hidden state and cell state?” (Question 244).

Implementation detail: PyTorch computes the GRU candidate with the reset gate applied after the recurrent linear transformation, whereas the original GRU equations above apply it before. The one-dimensional example with zero candidate bias yields the same number either way, but multidimensional implementations may differ.

Key takeaway: Reset controls what past information influences the proposed update; update determines how much old information survives in the final state.

Research: Cho et al. (2014), *Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation*. Implementation/reference: PyTorch `torch.nn.GRU` and Dive into Deep Learning — GRU.