Numerical Stability in Practice

Log-sum-exp, stable softmax, gradient clipping, and mixed-precision loss scaling. The tricks that keep exponentials from overflowing and gradients from exploding in real training runs.

12 problems