SIAM News Blog
SIAM News

# Data-enabled, Physics-constrained Predictive Modeling of Complex Systems

Even with recent advances in computing power and algorithms, a multitude of first-principles-based computations of physical problems remains out of reach, even on the most powerful supercomputers. This situation is unlikely to change for many decades for a number of important problems, such as those that involve bridging the gap between atomistic and continuum scales in materials science and combustion, climate and weather prediction, and structure formation and evolution in the universe. As a result, the scientific community continues to rely on physical intuition and empiricism to derive approximate models for prediction and control.

With the proliferation of high-resolution datasets and advances in computing and algorithms over the past decade, data science has risen as a discipline in its own right. Machine learning-driven models have attained spectacular success in fields such as language translation [3], speech and face recognition, bioinformatics, and advertising. The natural question to ask then is: Can we bypass the traditional ways of intuition/hypothesis-driven model creation and instead use data to generate predictions of physical problems? In other words, can one extract cause-and-effect relationships and create reliable predictive models based on a large number of observations of physical phenomena? The instinctive response from the physical modeler is typically along the lines of “curve-fitting is not physics,” and the like. The data scientist or statistician, on the other hand, exudes more optimism based on faith in approximation theory, reinforced by successes in some application domains. Though application has been restricted to simple problems, inference and machine learning have indeed been used to extract operator matrices [6], discover dynamical systems [1, 10], and derive the solution of differential equations [7]. To develop improved predictive models of complex real-world problems, however, one may need to pursue a more balanced view. Data cannot be an alternative for physical modeling, but when combined with—and informed by—a detailed knowledge of the physical problem and problem-specific constraints, it is likely to yield successful solutions.

Consider a true process $$\mathbf{u}$$ governed by a set of equations that are either unknown or too expensive to discretize. Assuming we want to derive data-driven models for a surrogate variable $$\mathbf{\bar{u}}$$, a few  challenges  arise: there  may  exist  several  latent  variables $$\mathbf{\bar{v}}$$ (which may encode the relationship between $$\mathbf {u}$$ and $$\mathbf{\bar{u}}$$) that might not be identifiable without a knowledge of the physics; we may not have enough data in all regimes of interest; and the data may be noisy and of variable quality. As a result of these challenges, if one were to apply machine learning directly to datasets (as in a speech recognition/translation example), the validity of the resulting models may be limited around the specific circumstances that generated the data. In other words, the predictive model might not be generalizable. Therefore, a pragmatic solution is to combine physics-based models with data-based methods and pursue a hybrid approach. The rationale is as follows: by blending data with existing physical knowledge and enforcing known physical constraints [4] (mass and energy conservation, for instance), one can improve model robustness and consistency with physical laws and address gaps in data. This would constitute a data-augmented physics-based modeling paradigm.

In traditional modeling approaches, it is typical to write down a set of physics-based governing equations $$\mathcal{M}(\mathbf{\bar{u}},\mathbf{\bar{v}})=0$$, where $$\mathcal{M}$$ is a model operator. Using this model as a starting point, data-augmented models may seek a model form$$\mathcal{M}^\gamma(\mathbf{\bar{u}},\mathbf{\bar{v}},\boldsymbol{\beta}(\mathbf{\bar{u}},\mathbf{\bar{v}}),\boldsymbol{\alpha})=0$$, where $$\boldsymbol{\alpha}$$ may be a set of parameters, $$\boldsymbol{\beta}$$ may be a functional form, and $$\mathcal{M}^\gamma$$ may be an augmented set of operators, likely derived based on available data and selection of priors.

In some applications, the use of machine learning can be direct and powerful. For instance, researchers have used manifold learning to extract a constitutive relationship in solid mechanics directly from data [2]. They have successfully employed machine learning to learn density functionals [9] in electronic structure calculations. In many problems, however, the information extracted from the data can be defined only in the context of the model. This aspect can restrict the use of machine learning on the data, even if the starting point is a physics-based model. As an example, a constitutive relationship $$\boldsymbol{\beta}(\mathbf{\bar{u}}_{data},\mathbf{\bar{v}}_{data})$$ extracted directly from the data may become inconsistent with the rest of the model in a predictive modeling setting because the latent variable in the model $$\mathbf{\bar{v}}_{model}$$ might be an operational variable, whereas its counterpart in the data $$\mathbf{\bar{v}}_{data}$$ might be a physical  variable. Consequently, machine learning must be preceded by statistical inference in many problems. Statistical inference thus uses data to derive problem-specific information that consistently connects model augmentations to model variables. If the goal is predictive rather than reconstructive modeling, the outputs of several such inverse problems (on different datasets representative of the physical phenomena) must be transformed into general functional forms $$\boldsymbol{\beta}(\mathbf{\bar{u}},\mathbf{\bar{v}})$$ using machine learning [5]. Figure 1 shows an example where lift data from one airfoil is used to infer and reconstruct functional forms of turbulence model discrepancies on other airfoils at flow conditions different from those in which the model was trained.

Figure 1. Example of data-augmented, physics-based modeling applied to turbulent flow prediction. Predictive improvement is achieved based on inferring force data over another airfoil and constructing machine-learned model augmentations. Left. Pressure over airfoil surface. Green: baseline physics model. Red: machine learning-augmented physics model. Blue: experimental measurements. Middle. Baseline flow prediction (pressure contours and streamlines). Right. Flow prediction using machine learning-augmented physics model. Image adapted from [8].

Combining physical models with data-driven techniques yields the potential to derive predictive and generalizable models in a number of disciplines. While one can argue that model creation has always been data-enabled, renewed opportunities exist, courtesy of access to a high quality and quantity of data, better tools/techniques, and more powerful computers.

Against this backdrop, the scientific computing community is beginning to recognize the promise of data-enabled modeling of complex physics. Initial successes have been noted, albeit on academic problems. For these techniques to have a significant impact in application domains, scientific rigor in data preparation, selection of priors, feature engineering, model training, and model deployment must be combined with domain expertise.

References
[1] Brunton, S.L., Proctor, J.L., & Kutz, J.N. (2016). Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the National Academy of Sciences, 113(15), 3932-3937.
[2] Ibañez, R., Abisset-Chavanne, E., Aguado, J.V., Gonzalez, D., Cueto, E., & Chinesta, F. (2016). A Manifold Learning Approach to Data-Driven Computational Elasticity and Inelasticity. Archives of Computational Methods in Engineering, 1-11.
[3] Lewis-Kraus, G. (2016, Dec. 14). The Great AI Awakening. The New York Times Magazine.
[4] Ling, J., Jones, R., & Templeton, J. (2016). Machine learning strategies for systems with invariance properties. Journal of Computational Physics, 318, 22-35.
[5] Parish, E.J., & Duraisamy, K. (2016). A paradigm for data-driven predictive modeling using field inversion and machine learning. Journal of Computational Physics, 305, 758-774.
[6] Peherstorfer, B., & Willcox, K. (2016). Data-driven operator inference for nonintrusive projection-based model reduction. Computer Methods in Applied Mechanics and Engineering, 306, 196-215.
[7] Raissi, M., Perdikaris, P., & Karniadakis, G.E. (2017). Inferring solutions of differential equations using noisy multi-fidelity data. Journal of Computational Physics, 335, 736-746.
[8] Singh, A.P., Medida, S., & Duraisamy, K. (2017). Machine-Learning-Augmented Predictive Modeling of Turbulent Separated Flows over Airfoils. AIAA Journal. Ahead of print, 1-13.
[9] Snyder, J.C., Rupp, M., Hansen, K., Müller, K.-R., & Burke, K. (2012). Finding Density Functionals with Machine Learning. Phys. Rev. Lett., 108.
[10] Williams, M.O., Kevrekidis, I.G., & Rowley, C.W. (2015). A data-driven approximation of the koopman operator: Extending dynamic mode decomposition. Journal of Nonlinear Science, 25(6), 1307-1346.

Karthik Duraisamy is an associate professor of aerospace engineering and director of the Center for Data-driven Computational Physics at the University of Michigan, Ann Arbor.