Optimizers and Surfaces

Visualizing the training process over various optimization surfaces. – Nov 12, 2024

Training a Machine Learning model consists of traverse an optimization surface, seeking the best solution: the configuration of parameters θ associated to the lowest error or best fit. It would be interesting to have a have a visual representation of this process, in which problems such as local minima, noise and overshooting are explicit and better understood.