In this talk we explore some new convergence results for stochastic gradient descent which suggest some interesting selection strategies. We make some connections to the Kaczmarz method for solving systems of linear equations, and discuss the implications of these results to this setting, as well as some other variants of the method. We conclude with some experiments and interesting open problems.
Copyright. All Rights Reserved.