Addressing Challenges in Reduced-Order Modeling

By Kevin Carlberg

One of applied mathematics’ great contributions is the foundation it provides for simulating physical phenomena. From the derivation of consistent, stable, and convergent discretization schemes to the development of efficient parallel solvers, mathematical advances have enabled the ubiquitous nature of modeling and simulation in applications ranging from protein-structure prediction to aircraft design. Today, the predictive capability of validated computational models allows simulation to replace physical experimentation in many scenarios, which facilitates the realization of deeper analyses and better designs at lower costs. However, there is a catch: the resolution required to achieve such high fidelity leads to large-scale models whose simulations can consume weeks on a supercomputer. This creates a massive gap between the simulation times of high-fidelity models and the rapid time-to-solution demands of time-critical (e.g., real-time analysis) and many-query (e.g., uncertainty quantification) applications in engineering and science.

To bridge this gap, researchers have pursued reduced-order modeling—which integrates techniques from data science, modeling, and simulation—as a strategy for reducing the computational cost of such models while preserving high levels of fidelity. First, these methods execute analyses (e.g., simulating the model, solving Lyapunov equations) during an offline ‘training’ stage; these analyses generate data that  can be mined to extract important physical features, such as low-dimensional solution manifolds and interpolation points for approximating nonlinear functions. Next, these techniques reduce the dimensionality and computational complexity of the high-fidelity model by projecting the governing equations onto the low-dimensional manifold and introducing other approximations where necessary. The resulting reduced-order model (ROM) can then be rapidly simulated during an online ‘deployed’ stage.

While significant advances have been made in reduced-order modeling over the past fifteen years, many outstanding challenges face the community, especially with respect to applying model reduction to parameterized nonlinear dynamical systems. The West Coast ROM Workshop—held last November at Sandia National Laboratories in Livermore, California—brought together local researchers from both academia and the national laboratories to address these challenges. Speakers provided a range of interesting perspectives on the topic.

One workshop theme focused on applying ROMs to truly large-scale nonlinear problems in engineering and science. To motivate this, invited speaker Charbel Farhat provided a number of compelling examples in which the computational cost incurred by such models poses a major bottleneck to design engineers across the naval, aerospace, and automotive industries. A number of challenges arise in this case. First, ROM techniques must be tightly integrated with the original high-fidelity simulation code because most nonlinear ROM methods realize computational savings by performing computations with the high-fidelity model on a small subset of the computational domain. Second, ensuring accurate ROM solutions can be challenging due to the complex dynamics (e.g., stiffness, chaoticity) exhibited by many large-scale dynamical systems. Finally, when the model is very large scale, the computational costs of both the offline training and online deployment can remain prohibitive; devising ways to reduce them is often essential.

Farhat presented promising results for the energy-conserving sampling and weighting (ECSW) ROM method [5] applied to large-scale problems in structural dynamics (Figure 1). This method integrates with finite-element codes by performing potential-energy computations on a subset of the domain’s elements; it produces accurate solutions by ensuring the ROM inherits the energy-conservation principle of the high-fidelity model. Jeffrey Fike presented results on integrating nonlinear ROM methods with a compressible fluid-dynamics code (developed at Sandia), and Irina Tezaur proposed a supporting approach for improving the accuracy of compressible-flow ROMs by rotating typical solution subspaces to include modes needed for energy dissipation [1]. Kyle Washabaugh presented an approach for robustly deforming a sample mesh in HPC environments when using ROMs to predict steady-state aerodynamic flows subjected to geometric deformations [11]. In contrast, invited speaker J. Nathan Kutz offered a framework that enables nonlinear model reduction without requiring access to the high-fidelity simulation code. The approach, based on Koopman theory, approximates nonlinear dynamical systems with a linear operator constructed from time-history data of dynamical-system observables [2]. To reduce offline training costs, Geoffrey Oxberry presented a method that leverages ideas from error control in adaptive time integrators to decrease the amount of data needed to construct low-dimensional solution subspaces [9]. To decrease online simulation times in parallel-computing environments, Kevin Carlberg proposed a new method for time parallelism. This technique applies data-driven ROM-solution forecasts [3] as a coarse propagator in the parareal framework.

Figure 1. Simulating the underbody blast of a V-hull vehicle with a ROM. Original computational domain with 2.4 × 105 elements (left), subset of the domain with 2 × 103 elements used by the ROM (center), and a comparison of the results (right). The ROM generated sub-1% displacement errors with a 104 wall-time speedup. Courtesy C. Farhat.

A second major workshop theme focused on applying ROMs to design optimization.  These many-query problems—which are often formulated as mathematical optimization problems constrained by partial differential equations—can require hundreds of simulations (and sensitivity analyses) of the computational model. Thus, rapid model evaluations are necessary when faced with time or resource constraints. Youngsoo Choi presented one approach for applying ROMs to design optimization [4]. The method adopts the classical offline–online strategy, wherein a database of ROMs for the (linear) model is constructed offline, and these ROMs are interpolated online on appropriate matrix manifolds. This method is amenable to real-time applications because it does not require any online high-fidelity simulations; however, it lacks convergence guarantees  and—due to the costly offline stage—requires the number of online optimization  iterations to exceed  a ‘break-even’ threshold before computational savings can be realized. Invited speaker Louis Durlofsky proposed a related method based on the trajectory piecewise linear (TPWL) ROM, and showed promising results on oil-production optimization under water injection [6]. Matthew Zahr proposed an alternative approach [12] that eschews the typical offline–online strategy in favor of a trust-region approach, wherein the high-fidelity-model solution (and sensitivities) are computed at trust-region centers (Figure 2). This approach guarantees convergence, but is not amenable to real-time applications due to the ‘mixing’ of high-fidelity and reduced-order model evaluations during the solution to the optimization problem. Both Zahr and invited speaker Michael Frenklach proposed strategies for reducing the dimensionality of high-dimensional parameter spaces; the former employed an adaptive strategy based on the gradient of the Lagrangian, while the latter applied active subspaces to a combustion-chemistry problem. 

Other important topics were addressed, with contributions from Tanya Kostova, who presented a technique employing both the system state and velocity as data in solution-subspace computation [8]; Sumeet Trehan, who has applied statistical learning to construct TPWL error surrogates; Syuzanna Sargsyan, who has  developed ROMs that adapt to particular physical regimes [10]; and invited speaker Jaijeet Roychowdhury, who proposed a representation of continuous systems as boolean finite state machines [7].

Figure 2. Topology optimization with reduced-order models. Optimal solution (left), ROM-based optimization solution after 2000 seconds (center), high-fidelity-model-based optimization after 2000 seconds (right). Courtesy M. Zahr.

Despite the many challenges, model reduction remains an exciting research area that is making rapid progress toward bridging the gap between high-fidelity models and time-critical applications in engineering and science.

References
[1] Balajewicz, M., Tezaur, I., & Dowell, E. (2015). Minimal subspace rotation on the Stiefel manifold for stabilization and enhancement of projection-based reduced order models for the compressible Navier- Stokes equations. Cornell University Library. Preprint, arXiv:1504.06661.

[2] Brunton, S.L., Brunton, B.W., Proctor, J.L., & Kutz, J.N. (2015). Koopman observable sub-spaces and finite linear representations of nonlinear dynamical systems for control. Cornell University Library. Preprint, arXiv:1510.03007.

[3] Carlberg, K., Ray, J., & Van Bloemen Waanders, B. (2015). Decreasing the temporal complexity for non- linear, implicit reduced-order models by forecasting. Computer Methods in Applied Mechanics and Engineering, 289, 79–103.

[4] Choi, Y., Amsallem, D., & Farhat, C. (2015). Gradient-based constrained optimization using a database of linear reduced-order models. Cornell University Library. Preprint, arXiv:1506.07849.

[5] Farhat, C., Avery, P., Chapman, T., & Cortial, J. (2014). Dimensional reduction of nonlinear finite element dynamic models with finite rotations and energy-based mesh sampling and weighting for computational efficiency. International Journal for Numerical Methods in Engineering, 98(9), 625–662.

[6] He, J., & Durlofsky, L.J. (2014). Reduced-order modeling for compositional simulation by use of trajectory piecewise linearization. SPE Journal, 19(05), 858–872.

[7] Karthik, A.V., Ray, S., Nuzzo, P., Mishchenko, A., Brayton, R., & Roychowdhury, J. (2014). ABCD-NL: Approximating continuous non-linear dynamical systems using purely boolean models for analog/mixed- signal verification. Design Automation Conference (ASP-DAC), 2014 19th Asia and South Pacific (pp. 250-255). Singapore: IEEE.

[8] Kostova, T., Oxberry, G., Chand, K., & Arrighi, W. (2015). Error bounds and analysis of proper orthogonal decomposition model reduction methods using snapshots from the solution and the time derivatives. Cornell University Library. Preprint, arXiv:1501.02004.

[9] Oxberry, G., Kostova-Vassilevska, T., Arrighi, B., & Chand, K. (2015). Limited-memory adaptive snapshot selection for proper orthogonal decomposition. Livermore, CA: Lawrence Livermore National Laboratory (LLNL).

[10] Sargsyan, S., Brunton, S.L., & Kutz, J.N. (2015). Nonlinear model reduction for dynamical systems using sparse sensor locations from learned libraries. Physical Review E, 92(3), 033304.

[11] Washabaugh, K., Zahr, M.J., & Farhat, C. (2016). On the Use of Discrete Nonlinear Reduced-Order Models for the Prediction of Steady-State Flows Past Parametrically Deformed Complex Geometries. 54th AIAA Aerospace Sciences Meeting (p. 1814). San Diego, CA: AIAA SciTech.

[12] Zahr, M.J., & Farhat, C. (2015). Progressive construction of a parametric reduced-order model for PDE-constrained optimization. International Journal for Numerical Methods in Engineering, 102(5), 1111–1135.