Research

ImageNet Classification with Deep Convolutional Neural Networks (AlexNet)

AlexNet showed that deep convolutional networks trained on large image datasets with GPUs could dramatically improve image recognition.

Alex Krizhevsky, Ilya Sutskever, Geoffrey E. HintonDec 3, 2012Advances in Neural Information Processing Systems 25 (NeurIPS 2012)12 min readUniversity of Toronto
TRENDINGDeep LearningIntermediate

Key takeaway

AlexNet combined a deep CNN, ReLU activations, GPU training, data augmentation and dropout to achieve a major ImageNet breakthrough. Its 2012 competition model reached 15.3% top-5 test error, compared with 26.2% for the second-best entry.

Updated on Nov 17, 2026

01

Abstract

We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully-connected layers we employed a recently-developed regularization method called “dropout” that proved to be very effective.

02

Methodology

The Core Idea

Instead of manually engineering visual features, AlexNet learns a hierarchy directly from pixels. Early convolutional layers detect simple patterns such as edges and textures; deeper layers combine them into more complex shapes and object-level representations. A final classifier maps the learned representation to 1,000 ImageNet categories.

Figure 1

03

Architecture

AlexNet Architecture

The network has eight learned layers: five convolutional layers followed by three fully connected layers. ReLU activations are applied throughout; max pooling reduces spatial dimensions; the final 1,000-way softmax produces class probabilities. The original design used two GPUs, with selected layers split across them. The network contains about 60 million parameters.

04

Training

How to Reproduce the Core Idea Today

Use a standard image-classification dataset and a framework such as PyTorch. Build five convolutional stages, ReLU activations, pooling and a classifier; train with cross-entropy and SGD with momentum; apply random crops, horizontal flips and channel normalization. Track top-1/top-5 accuracy, training curves and validation overfitting. A modern reproduction should document deviations from the original preprocessing, hardware and optimizer settings.

05

Results

Results and Benchmark Context

06

Limitations

Limitations and What Changed Later

AlexNet was expensive by today's standards, had a large fully connected parameter count, and relied on a fixed 1,000-class classification setup. Local response normalization and several architectural choices were later superseded. VGG, GoogLeNet and ResNet subsequently explored deeper or more efficient designs. AlexNet remains historically important, not a modern default architecture.

07

Conclusion

Key Takeaways

AlexNet made a compelling case for scaling CNNs with data and compute. ReLU improved optimization speed; augmentation and dropout controlled overfitting; GPU parallelism enabled a much larger model. Its enduring lesson is that architecture, data, optimization and systems engineering must be considered together.

Browse all research