VisualizationsOptimization
Gradient Descent
Watch an optimizer roll down a loss surface. Tune the learning rate and momentum, drop a starting point, and see it converge or diverge.
Click anywhere to drop a new starting point.
Brighter = higher loss. Watch how a high learning rate overshoots, and how momentum powers through the small bumps toward a minimum.
Related topics: Optimization