Preconditioned gradient descent

written by Theory Club

published November 18, 2022

Stephen lectured on different forms of preconditioning gradient descent including the natural gradient and Newton’s method for convex problems.

Some related papers:

• https://arxiv.org/abs/1306.0895
• https://arxiv.org/abs/1704.02315
• https://arxiv.org/abs/1704.02310