Research

Deep Residual Learning for Image Recognition: The Paper That Made Very Deep Networks Trainable

ResNet introduced residual connections that let layers learn a correction to their input, enabling much deeper convolutional networks to optimize effectively.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian SunDec 10, 2015Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016)14 min readMicrosoft Research
FOUNDATIONALComputer VisionIntermediate

Key takeaway

Instead of asking stacked layers to learn a complete mapping H(x), ResNet asks them to learn a residual F(x)=H(x)−x and adds the input back: y=F(x)+x. This shortcut creates a direct information and gradient path through the network. The paper showed that residual networks were easier to optimize than plain networks and achieved strong results with depths up to 152 layers on ImageNet. Its central lesson: make identity mappings easy to represent, so additional depth can learn useful refinements rather than having to relearn everything.

Updated on Nov 17, 2026

01

Abstract

02

Methodology

The Core Idea: Learn the Residual

Let H(x) be the desired mapping. A conventional stack of layers tries to learn H(x) directly. ResNet instead parameterizes the mapping as F(x) = H(x) − x. Therefore H(x) = F(x) + x.

The nonlinear layers learn F(x), while a shortcut passes x around those layers and adds it to their output. If the best transformation is close to identity, the residual branch can learn a value close to zero. This can be easier than forcing several nonlinear layers to approximate an identity mapping from scratch.

Figure 1

Residual learning: the original building block

Original Figure 2 from the paper. The input x bypasses two weight layers through an identity shortcut and is added to the learned residual F(x), producing F(x)+x before the final ReLU.

He et al., Deep Residual Learning for Image Recognition, Figure 2, arXiv:1512.03385.

03

Architecture

Inside a Residual Block

The paper’s basic residual building block stacks two 3×3 convolutional layers. The input x travels through the residual branch, while an identity shortcut bypasses those layers. The two paths are combined by element-wise addition, followed by a ReLU activation in the illustrated basic block.

For a residual block with matching dimensions, the shortcut is simply the identity and adds no parameters or meaningful computational cost. If the input and output dimensions differ, the shortcut must align them. The paper studies zero-padding identity shortcuts and learned projection shortcuts, commonly implemented with a 1×1 convolution.

The paper also introduces a bottleneck design for deeper networks: a 1×1 convolution reduces channels, a 3×3 convolution processes them, and a final 1×1 convolution expands channels. This reduces computation while enabling ResNet-50, ResNet-101 and ResNet-152.

04

Training

How to Implement a Residual Block

A minimal PyTorch-style basic residual block uses two convolutions and adds the input back. If the spatial dimensions or channel count change, use a projection shortcut to align the shapes.

Implementation outline:

  1. Compute the residual branch with Conv2d → BatchNorm → ReLU → Conv2d → BatchNorm.
  2. Compute the shortcut as identity when shapes match; otherwise use a 1×1 convolution (and normalization) with the required stride.
  3. Add residual and shortcut tensors.
  4. Apply ReLU to the sum.

In code, the core forward operation is: identity = shortcut(x); out = bn2(conv2(relu(bn1(conv1(x))))); return relu(out + identity).

The code illustrates the basic residual block. It is not a full reproduction of the paper’s ImageNet training recipe, which also specifies data augmentation, initialization, batch normalization, SGD, learning-rate scheduling and multi-crop or multi-scale evaluation.

05

Results

Experiments and ImageNet Results

The paper evaluates ImageNet 2012 classification, CIFAR-10 and transfer to other recognition tasks. On ImageNet, a 34-layer plain network had 28.54% top-1 error under the paper’s 10-crop validation protocol, while its 34-layer residual counterpart achieved 25.03%. The 34-layer residual network also had lower training error, directly addressing the degradation phenomenon.

The single-model ImageNet validation table reports 19.38% top-1 and 4.49% top-5 error for ResNet-152. Separately, the paper reports that an ensemble achieved 3.57% top-5 error on the ImageNet test set and won the ILSVRC 2015 classification task. These figures refer to different evaluation settings and should not be conflated.

The authors also trained models over 100 layers on CIFAR-10 and explored 1000-layer networks. The paper reports a 28% relative improvement on COCO object detection from the deeper representations. The residual networks also supported first-place results in several ILSVRC and COCO 2015 tasks.

Result board

34-layer plain

28.54% top-1 error34-layer ResNet: 25.03% in the reported 10-crop validation comparison

ResNet-152 single model

19.38% top-1 and 4.49% top-5 validation error

Ensemble

3.57% top-5 test error; ILSVRC 2015 classification winner

Result

The paper also studies CIFAR-10 models with 100 and 1000 layers

Result

Reported 28% relative improvement on COCO object detection

06

Limitations

Limitations and Design Trade-offs

Residual connections do not make depth free. Deeper networks still require more memory, compute and training time, and the architecture must manage tensor dimensions at shortcut merges. Projection shortcuts introduce parameters and computation when shapes change.

The original paper focuses on supervised visual recognition, primarily convolutional architectures. Its experiments do not establish that every task or network benefits equally from increasing depth. Very deep networks can still be difficult to train and deploy, and later work explored pre-activation residual units, improved normalization, efficient scaling and alternative architectures.

07

Conclusion

Key Takeaways

ResNet changed deep-network design by making identity mappings easy to express. A block learns a residual correction and adds the original input back. This small architectural change made much deeper networks easier to optimize and became one of the most influential ideas in modern neural-network architecture.

The general pattern—transform a representation, then preserve a direct path around the transformation—appears in many later architectures. The original paper’s result should be understood in its experimental context: residual learning improved optimization and enabled strong very-deep visual recognition models.

Browse all research