An introduction to Scientific Machine Learning
Andrew Christlieb
Michigan State University
Department of Mathematics and Department of Computational Mathematics, Science and Engineering
Scientific machine learning is a fundamental tool for enabling the design and control of plasmas in ways that have not been possible until now. As a rapidly developing subject area, innovation in the design of methods can have a dramatic impact on the outcomes produced by models. These models are being used as surrogates to bridge scales for non-LTE plasmas in blended computing, as surrogates in optimal design, and in the direct construction of solutions to PDEs themselves.
Blended computing refers to combining ML models with traditional numerical methods: accurate collision operators, closure models for hyperbolic systems, incorporation of radiation effects through efficient surrogates, etc. Direct numerical simulation with neural networks might involve long-time integration of Hamiltonian systems or simulation of high-dimensional PDEs such as the Radiation Transport Equations or the Vlasov–Poisson system.
I will start with a broad overview of ML in plasma science, followed by a short review of neural networks. Next, I will introduce the two main strategies for doing this kind of work and work through accessible examples: structure-informed neural networks and structure-preserving neural networks.
Structure-informed neural networks encode physics in the cost function of the NN in the form of the residual. By driving the error to zero during training, the NN is forced to approximate a solution to the PDE encoded in the cost function at the training points. This approach is known as Physics-Informed Neural Networks (PINNs).
Structure-preserving neural networks incorporate inductive biases in the direct construction of the NN, thereby enforcing that the mathematical structure is provably preserved. This type of NN can approximate solutions far outside of a training window and ensure that the physics encoded in the NN is preserved indefinitely. The former approach is more flexible, while the latter serves as a useful method for creating surrogates for robust simulation tools when constructing hybrid models.
When discussing PINNs, I will walk the group through an example using the Poisson equation and touch on state-of-the-art methods for creating effective models, including recent work using the SOAP optimizer.
When discussing structure-preserving neural networks, I will walk through the concrete example of approximating Hamiltonian systems with a provably symplectic neural network. As a way of thinking about how to construct a structure-preserving NN, I will explain what it means for a system to be Hamiltonian, why a symplectic map is a good idea for constructing a time integrator, and how these ideas translate into building a symplectic neural network.
This work is supported by the Department of Energy MMICCs center – Center for Hieratical and Robust Modeling of Non-Equilibrium Transport (CHaRMNET) and the Department of Energy PSAAP- Focused Investigatory Center - High Order Plasma Turbulence Modeling for Z-Pinch (HighZ).
