Workshop II: Theory and Practice of Deep Learning - IPAM

Workshop II: Theory and Practice of Deep Learning

Part of the Long Program Mathematics of Intelligences
October 14 - 18, 2024

Schedule

All times in this Schedule are Pacific Time (PT)

Monday, October 14, 2024

Morning Session

08:00-08:55
Check-In/Breakfast (Hosted by IPAM)
08:55-09:00
Welcome and Opening Remarks
09:00-09:00
SESSION CHAIR: Jacob Foster
09:00-09:50
Boris Hanin (Princeton University)
Neural Network Scaling Limits
10:00-10:15
Break
10:15-11:05
Wu Lin (Vector Institute)
Structured Matrix Inverse-free Adaptive Methods for Large Neural Networks
11:15-11:30
Break
11:30-12:20
Nikos Tsilivis (New York University)
The Price of Implicit Bias in Robust ML

Afternoon Session

12:30-14:30
Lunch (on your own)
14:30-14:30
SESSION CHAIR: Jacob Foster
14:30-15:20
Elvis Dohmatob (Facebook AI Research)
The Mathematics of Scaling Laws and Model Collapse in AI
15:30-15:45
Break
15:45-16:15
Lightning Poster Session
16:15-18:30
Poster Session & Reception (Hosted by IPAM)

Tuesday, October 15, 2024

Morning Session

08:00-09:00
Check-In/Breakfast (Hosted by IPAM)
09:00-09:00
SESSION CHAIR: Boris Hanin
09:00-09:50
Cengiz Pehlevan (Harvard University)
Asymptotic theory of in-context learning by linear attention
10:00-10:15
Break
10:15-11:05
Gintare Karolina Dziugaite (Google DeepMind)
The dynamics of memorization and generalization in deep learning
11:15-11:30
Break
11:30-12:20
Vidya Muthukumar (Georgia Institute of Technology)
Comparison and transfer between tasks in overparameterized learning

Afternoon Session

12:30-14:30
Lunch (on your own)
14:30-14:30
SESSION CHAIR: Boris Hanin
14:30-15:20
Sam Smith (Google DeepMind)
How to train an LLM
15:30-16:00
Break
16:00-16:50
Patrick Shafto (Rutgers University)
Common Ground in Cooperative Communication

Wednesday, October 16, 2024

Morning Session

08:00-09:00
Check-In/Breakfast (Hosted by IPAM)
09:00-09:00
SESSION CHAIR: Patrick Shafto
09:00-09:50
Daniel Roy (University of Toronto)
The Size of Teachers as a Measure of Data Complexity: PAC-Bayes Excess Risk Bounds and Scaling Laws
10:00-10:15
Break
10:15-11:05
Shaowei Lin (Topos Institute)
Singular Learning, Relative Information and the Dual Numbers
11:15-11:30
Break
11:30-12:20
Paul Riechers (Simplex)
Sometimes fractal and sometimes quantum: Universal geometric representations of beliefs about the far future in deep neural networks pretrained on next-token prediction

Afternoon Session

12:30-12:30
Group Photo
12:30-14:30
Lunch (on your own)
14:30-14:30
SESSION CHAIR: Patrick Shafto
14:30-15:20
Blake Bordelon (Harvard University)
Infinite limits and scaling laws of neural networks
15:30-16:00
Break
16:00-16:50
Discussion

Thursday, October 17, 2024

Morning Session

08:00-09:00
Check-In/Breakfast (Hosted by IPAM)
09:00-09:00
SESSION CHAIR: Misha Belkin
09:00-09:50
Mauro Maggioni (John Hopkins University)
On exploiting compositional structure: one bit of theory and one application.
10:00-10:15
Break
10:15-11:05
Dmitry Krotov (Massachusetts Institute of Technology)
Generative AI models through the lens of Dense Associative Memory
11:15-11:30
Break
11:30-12:20
Fanny Yang (ETH Zurich)
Surprising phenomena of max-lp-margin classifiers in high dimensions

Afternoon Session

12:30-14:30
Lunch (on your own)
14:30-14:30
SESSION CHAIR: Misha Belkin
14:30-15:20
Oliver Eberle (Technische Universität Berlin)
Interpretability for Deep Learning: Theory, Applications and Scientific Insights
15:30-16:00
Break
16:00-16:50
Mayank Mehta (University of California, Los Angeles (UCLA))
Dynamics of brain's deep network

Friday, October 18, 2024

Morning Session

08:00-09:00
Check-In/Breakfast (Hosted by IPAM)
09:00-09:00
SESSION CHAIR: Monty Abbas
09:00-09:50
Adityanarayanan Radhakrishnan (Broad Institute)
How do neural networks learn features from data?
10:00-10:15
Break
10:15-11:05
Leena Vankadara (Amazon Research)
Beyond muP: Scaling Insights from Infinite-Width Theory for Next Generation Architectures and Learning Paradigms
11:15-11:30
Break
11:30-12:20
Misha Belkin (University of California, San Diego (UCSD))
Emergence and grokking in simple architectures

Afternoon Session

12:30-14:30
Lunch (on your own)
Printable Version