In this work we develop two mathematically principled deep learning algorithms for learning unknown governing equations from trajectory data that are completely data-driven, robust to noise and can be used to solve a variety of real-world problems coming from areas such as economics, biology, and finance. Given samples of solutions x(t) to an unknown dynamical system x’(t) = f(t,x(t)) we approximate the function f using a neural network. We provide two main contributions. The first one is the fact that we use a Lipschitz regularized neural network N to approximate the unknown function f. Specifically, we add a Lipschitz regularization term to our loss function to force the Lipschitz constant of the approximating network to be small. We empirically show that this regularization results in a smoother approximating function and better generalization properties when compared with non-regularized models, especially in presence of noise in the data. The target data used to train the network N are approximations of the velocity vector x’(t), which act as a prior for the unknown function f. The quality of the target data strongly influences the quality of the approximation of the function f so that a fundamental component of our model is the data processing and the generation of accurate target data. The second main novelty of this work is that we propose two techniques for the data preprocessing: one based on Splines and one based on a family of neural networks. In the examples, we show that both techniques result in models that can avoid overfitting and that are robust to noise. We show that, since the neural network preprocessing is based on weak notion of solution using integration it can be used to reconstruct differential equations for large amounts of noise in the data (up to 10%) and even for non-smooth RHS functions, while the spline-based method can only reconstruct differential equations with smooth solutions and noise amount up to 2%. Finally, we compare our methods with other methods for system identification.
Copyright. All Rights Reserved.