Machine Learning & Training

What Is the Exploding Gradient Problem?

The exploding gradient problem occurs when gradients grow uncontrollably large as they propagate through a deep network during backpropagation. This can cause model weights to update by huge amounts, leading to unstable or diverging training. Gradient clipping and careful weight initialization are common remedies.