Stochastic Gradient Methods - IPAM

Stochastic Gradient Methods

February 24 - 28, 2014

Schedule

All times in this Schedule are Pacific Time (PT)

Monday, February 24, 2014

Morning Session

08:00-08:50
Check-In/Light Breakfast (Hosted by IPAM)
08:50-09:00
Welcome and Opening Remarks
09:00-09:45
Nathan Srebro (TTI-Chicago)
Distributed Stochastic Optimization: The Role of SGD
10:00-10:15
Break
10:15-11:00
Ben Recht (University of California, Berkeley (UC Berkeley))
Why we should all run Hogwild!
11:15-11:30
Break
11:30-12:15
Yoram Singer (Google Inc.)
Adaptive Gradient (AdaGrad) and Generalized Accelerated Gradient Ascent (GAGA): two sides of the same coin?

Afternoon Session

12:30-14:30
Lunch (on your own)
14:30-15:15
John Langford (Microsoft Research)
Learning to Interact
15:30-16:00
Break
16:00-16:45
Alekh Agarwal (Microsoft Research)
Learning Sparsely Used Overcomplete Dictionaries
17:00-18:30
Poster Session & Reception (Hosted by IPAM)

Tuesday, February 25, 2014

Morning Session

08:00-09:00
Check-In/Breakfast (Hosted by IPAM)
09:00-09:45
Dimitri Bertsekas (Massachusetts Institute of Technology and Arizona State University)
Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: A Unified Framework
10:00-10:15
Break
10:15-11:00
Peter Richtarik (University of Edinburgh)
Accelerated, Parallel and Proximal Coordinate Descent
11:15-11:30
Break
11:30-12:15
Rachel Ward (University of Texas at Austin)
Stochastic Gradient Descent with Importance Sampling

Afternoon Session

12:30-14:30
Lunch (on your own)
14:30-15:15
Deanna Needell (University of California, Los Angeles (UCLA))
SGD and its connections to the Kaczmarz method
15:30-16:00
Break
16:00-16:45
Stephen Wright (University of Wisconsin-Madison)
Asynchronous parallel stochastic Kaczmarz and stochastic coordinate descent algorithms

Wednesday, February 26, 2014

Morning Session

08:00-09:00
Check-In/Breakfast (Hosted by IPAM)
09:00-09:45
Yann LeCun (New York University)
Open problems in large-scale stochastic optimization for deep learning
10:00-10:15
Break
10:15-11:00
Francis Bach (Institut National de Recherche en Informatique Automatique (INRIA))
Efficient and robust stochastic approximation through an online Newton method
11:15-11:30
Break
11:30-12:15
Jorge Nocedal (Northwestern University)
A Stochastic Quasi-Newton Method for Large Scale Learning

Afternoon Session

12:30-14:30
Lunch (on your own)
14:30-15:15
John Duchi (University of California, Berkeley (UC Berkeley))
Optimal rates for zero-order optimization: the power of two function evaluations
15:30-16:00
Break
16:00-16:45
Asuman Ozdaglar (Massachusetts Institute of Technology)
Distributed Alternating Direction Method of Multipliers for Multi-agent Optimization

Thursday, February 27, 2014

Morning Session

08:00-09:00
Check-In/Breakfast (Hosted by IPAM)
09:00-09:45
Mark Schmidt (Simon Fraser University)
Minimizing Finite Sums with the Stochastic Average Gradient
10:00-10:15
Break
10:15-11:00
Lin Xiao (Microsoft Research)
Proximal Stochastic Gradient Method with Variance Reduction
11:15-11:30
Break
11:30-12:15
David Blei (Columbia University)
Stochastic Variational Inference and Scalable Topic Models

Afternoon Session

12:30-14:30
Lunch (on your own)
14:30-15:15
David McAllester (TTI-Chicago)
A PAC-Bayesian Analysis of Dropouts
15:30-16:00
Break
16:00-16:45
Jason Weston (Facebook)
Embedding and ranking for images, entities, items and text

Friday, February 28, 2014

Morning Session

08:00-09:00
Check-In/Breakfast (Hosted by IPAM)
09:00-09:45
Steve Smale (City University of Hong Kong)
The protein folding problem: introduction and new results, exposition for mathematicians.
10:00-10:15
Break
10:15-11:00
Noboru Murata (Waseda University)
Semi-optimal on-line learning for restricted gradients
11:15-11:30
Break
11:30-12:15
James Spall (Johns Hopkins University)
Recent Advances in SPSA at the two Extremes: Adaptive Methods for Smooth Problems and Discrete Methods for Non-Smooth Problems

Afternoon Session

Conclusion
Printable Version