TL;DR
AlexNet showed that a deep convolutional network trained on GPUs could shatter the ImageNet benchmark, kicking off the deep-learning era in computer vision.
Key contributions
- A deep 8-layer CNN trained end-to-end on 1.2M images across 1000 classes
- ReLU activations that train several times faster than tanh/sigmoid
- Dropout as a powerful regularizer against overfitting
- A practical multi-GPU training recipe with data augmentation
How it works
- 1Five convolutional layers (some followed by max-pooling) feed three fully-connected layers ending in a 1000-way softmax.
- 2ReLU non-linearities keep gradients healthy and speed convergence dramatically.
- 3Overlapping max-pooling and local response normalization improve generalization.
- 4Dropout in the dense layers and aggressive augmentation (crops, flips, PCA color jitter) combat overfitting on 60M parameters.
Results
- Top-5 error of 15.3% on ILSVRC-2012 vs. 26.2% for the next best entry
- A decisive margin that made deep CNNs the obvious path forward
Why it mattered
AlexNet is widely credited with igniting the modern deep-learning revolution. Almost every vision system since builds on the CNN template it popularized, and the ReLU + dropout + augmentation recipe is still standard practice.
Key takeaways
- Depth + data + compute unlocks representation learning
- ReLU and dropout are simple ideas with outsized impact
- Benchmarks can catalyze whole fields when a method wins decisively