SGD and its connections to the Kaczmarz method

Deanna Needell
Claremont McKenna College

In this talk we explore some new convergence results for stochastic gradient descent which suggest some interesting selection strategies. We make some connections to the Kaczmarz method for solving systems of linear equations, and discuss the implications of these results to this setting, as well as some other variants of the method. We conclude with some experiments and interesting open problems.

Presentation (PDF File)

Back to Stochastic Gradient Methods