Training neural ODEs for density estimation

Christopher Finlay
McGill University
Mathematics and Statistics

Density estimation presents the following inverse problem: given a set of points sampled from an unknown probability distribution, infer the underlying distribution. In high dimensions, the technique of normalizing flows has had remarkable success tackling this problem. In this talk I will show how normalizing flows can be implemented using neural ODEs, rather than with traditional neural networks. I will present the problem of training neural ODEs through the lens of sensitivity analysis and optimal control. Moreover, I will present two regularization techniques, motivated by the dynamic formulation of Optimal Transport, which vastly speed up neural ODE training, and improve the stability of the learned ODE dynamics.

Presentation (PDF File)

Back to Workshop II: PDE and Inverse Problem Methods in Machine Learning