Data Assimilation for Personalized Treatment Strategies

By Melike Sirlanci, David J. Albers, and George Hripcsak

In areas of medicine with a deep body of available knowledge about relevant physiological systems, practitioners can purposely design treatments for diseases instead of relying on chance discovery. Under ideal circumstances, researchers would have a precise understanding of underlying mechanisms so that system models would perfectly represent reality; they would also have access to all necessary measurements at the required frequency and possess the correct level of computational power. All of these factors would then allow one to predict optimal personalized intervention strategies.

In most cases, however, current knowledge of human physiology, data, and other resources falls short of these ideal circumstances. Furthermore, human physiology is an emergent phenomenon that is established over many scales in space and time; full knowledge of the resulting system is therefore fanciful. As a result, researchers design treatments through either accidental discovery or the use of partial knowledge that are meant to work at a population level. They first test these treatments in the laboratory and eventually in clinical trials to assess their safety and efficacy. Such population-level treatments usually ignore inter- and intra-patient variability, and some patients do not respond to them. In order to provide better health care, next-generation medical treatments must outperform the current state of the art. They should be personalized, understandable, and driven by knowledge of and data from the generating system. The use of computational modeling techniques can help achieve this goal. One promising starting point is with well-modeled, data-driven systems and clearly defined problems like glycemic (i.e., glucose) management in the intensive care unit (ICU).

Glycemic management in the ICU—a crucial and challenging task—is a prototypical setting that can benefit from computational, personalized management strategies. Current glycemic management protocols in the ICU accommodate patient variability by monitoring measured glucose levels and current administration of insulin, but they fail to account for variations in patients’ responses to these inputs [12]. For example, management protocols to achieve tight glycemic control (TGC) in the ICU through intravenous (IV) insulin delivery have yielded contradictory results [6]. Some studies have found that TGC reduces mortality and morbidity in the ICU [13, 14], while others show either the exact opposite or no difference between the test and control groups [4, 5, 7, 10]. Researchers have proposed several different explanations for these contradictory results; the most widely accepted rationalization is that hypoglycemic episodes cloud the benefits of euglycemia. Indeed, hypoglycemia can cause various complications such as irreversible cardiac arrhythmia, neuronal damage, coma, and death. 

Figure 1. Researchers recently began to use mechanistic model-based approaches to test the efficacy of interventions via simulation, which provides a seamless opportunity to personalize interventions for optimal patient-level efficacy. However, these approaches suffer from various computational difficulties that stem from system behavior and data-related hurdles. Our methodology aims to address these difficulties and exploit the advantages of model-based techniques to develop systematized personalized intervention strategies. Figure courtesy of the authors.

Medical practitioners currently perform glycemic management by following flow charts, which is a labor-intensive process. In fact, some researchers claim that this procedure could distract health care providers and cause them to miss other, more pressing patient management needs. The use of model-based, human-in-the-loop glycemic controllers that avoid hypoglycemia as clinical decision support mechanisms can lessen the workload of health care providers, potentially improve the health status of ICU patients, and reduce time spent managing blood glucose (BG) levels.

So, how can we best personalize medical treatments? From a mathematics perspective, personalized treatments and computational methodological pipelines—once they are more fully developed—will likely generalize to other medical contexts with similar modeling, data, and information production requirements. Real application of these pipelines has remained elusive, and we return to the ICU setting to understand why.

Many computational challenges persist in the development of an effective model-based glycemic controller, most of which stem from challenges within the clinical context. Due to health changes and ICU interventions, patients' BG behaviors are extremely non-stationary above the typical inherited complexity of the glucose regulatory system [11]. Managing this non-stationarity is difficult because some processes that govern the system (like interventions) cannot be incorporated into the models in a straightforward way. Another challenging factor is the mathematical models’ unmeasured states. Nearly all models include dynamic state representations for plasma and interstitial insulin, which would ideally provide an increased physiological realism and fidelity to improve forecast and control accuracy. Yet clinicians never measure plasma or interstitial insulin in practice, which leaves them as free variables and makes the entire system non-identifiable. Additionally, it is not possible to differentiate endogenous and exogenous insulin in the blood from clinical measurements. The only measured state variable is generally BG, which is usually very sparse (at most 10 to 15 measurements a day). A continuous glucose monitor would decrease BG sparsity, but insulin would remain unmeasured. Though both the nutrition and exogenous insulin data are available, nutrition data are messy, incomplete, and reflect nutrition that is delivered to the stomach rather than nutrition that makes it into the blood. And while exogenous insulin data are reliable, they are also complex — one can deliver four classes of insulin via three different means. In this way, models must be disciplined by the existing data and their complexities rather than by a potentially more realistic idealized model.

To develop a model-based controller for glycemic management, we follow the data and create a model that is based on real data, is as accurate as possible, can be validated, and provides actionable information. The end goal is a computational pipeline for the ICU that health care providers can use as a decision support system and a human-in-the-loop control mechanism. Removing humans from the system is unrealistic due to current challenges related to the ICU, data availability, and nonstationarity. In addition, we want to take advantage of the unique information that is hidden in clinicians' experience and evaluations of patient health conditions — often called the health care process model [9]. In summary, we aim to directly exploit all available information and address the aforementioned challenges with a wide range of remedies. We do so in two steps.

First, we exploit the advantages of mechanistic modeling—which inherently involves some physiological knowledge of the system—while simultaneously making the model robustly estimable with 10 to 15 data points. There are two approaches for attacking this problem: (i) constrain the model or (ii) construct a simple model that manages uncertainty of unmeasured variables as lumped stochastic processes. Although constraining the model is an effective tool, constraints alone cannot ensure rapid and robust estimability for every patient [1, 3]. And in a health care setting, we are modeling physiological and health care process-generating processes rather than purely physiological processes [9]. In this case, adding fidelity while limiting the model to physiological processes can worsen problems with resolving the glycemic dynamics because of the unmeasured model states and health care procedures. Simpler models that appropriately parametrize unresolvable system components could provide better accuracy in forecasting and control-related goals.

Figure 2. Comparison of a virtual patient profile in two different scenarios for glycemic management: linear quadratic Gaussian (LQG) controller- and protocol-suggested insulin rates. The LQG controller outperforms the protocol by accounting for the patient's prior 24-hour data and delivered nutrition rate. Figure courtesy of the authors.

Second, we must be able to provide computable guidance: a task for control theoretic methods. The approaches for developing a controller depend on the model complexity — e.g., linear versus nonlinear and deterministic versus stochastic. For example, nonlinear models require a case-dependent specific investigation and substantially more data. However, a wide body of research has standardized the control of linear stochastic systems and made the process robust, a significant advantage over nonlinear stochastic models.

The backbones of our stochastic modeling approach are realism in modeling and control—which helps address computational difficulties—and control theoretical tools. These aspects reflect the system's basic physiological features, provide flexibility to account for limited nonlinear deterministic dynamics, and avoid computational issues.

We use a linear Gaussian stochastic differential equation model to represent the BG level dynamics of ICU patients [2]. This choice optimizes the model's fidelity level and allows us to resolve the dynamics that are active at a helpful level for clinical decision making — even given present computational challenges. The model's deterministic components reflect the body's effort to regulate glucose as well as nutrition and exogenous insulin’s effects on BG rate of change. The stochastic component accounts for unresolvable high-frequency fluctuations in BG levels. Training with patient data personalizes the model. We then utilize a linear quadratic Gaussian (LQG) controller that is based on this personalized model to estimate the personalized optimal rate of IV insulin to infuse into the patient. We do not aim to optimize the nutrition rate because clinicians generally maximize nutrition a priori to aid the healing process, but we do account for nutrition while estimating the optimal insulin rate.

To test the efficacy of our approach, we first simulated a cohort of virtual patients by creating realistic synthetic data with Tom Van Herpe’s ICU minimal model [8]. We employ a different model to simulate patients because simulating data with the same model that we use for forecasting or control would imply that we have a perfect representation of reality, which is not true. By exploiting the advantages of virtual patients, we then "deliver" the IV insulin rates that are suggested by the model-based controller and current glycemic management protocols and "observe" their effects via simulation. This comparison is essential because one of the controllers—the glycemic management protocol—is state of the art and currently applied in real-world settings. Figure 2 displays a comparison of protocol-suggested insulin and LQG controller-suggested insulin rates for a virtual patient. The LQG controller suggests slightly higher insulin rates based on the patient's data and outperforms the protocol by keeping a higher number of BG measurements within the target range. To compare the efficacy of controllers at this level, we rely on the percentage of BG measurements that are captured in the target region. When we consider all of the virtual patients, the average percentage is 71.92 percent for the LQG controller and 49.43 percent for the protocol.

Figure 3. Comparison of the linear quadratic Gaussian (LQG) controller- and protocol-suggested insulin rates for events of hypoglycemia and hyperglycemia. Figure courtesy of the authors.

We then utilize real patient data in a retrospective study. Because it was not possible to "deliver" and "observe" the effect of suggested optimal IV insulin rates to the patients, evaluation of the results required novel methods. We therefore compared the effects of model-based-controller- (LQG controller-) and protocol-suggested IV insulin rates for hypoglycemic and hyperglycemic episodes through the known insulin IV rate for each patient, since glycemic management aims to avoid these extreme events. We divided the possible scenarios into five different categories (see Figure 3). For example, if the protocol-suggested insulin rate is greater than or equal to the real insulin rate and the LQG controller-suggested insulin rate is less than the real insulin rate in case of hypoglycemia, this means that the LQG controller yielded a better insulin rate than the protocol. But if this scenario were a hyperglycemic event, the protocol would outperform the LQG controller. Comparison of the LQG controller and protocol against each other (second and third rows of Figure 3) indicates that the LQG controller outperformed the protocol both for hypoglycemic and hyperglycemic events. The difference in their performance in hypoglycemic events is more significant. Upon comparing the implications of hyperglycemic and hypoglycemic episodes for ICU patients, we determined that preventing all hypoglycemic events would be very helpful in glycemic management.

Our prototype for personalized glycemic management in the ICU demonstrates how personalized treatment strategies can increase success rates at the individual level. Because this framework also represents challenges that are present in many other biomedical settings, one could adopt our computational and modeling solution techniques accordingly. In summary, our personalized treatment strategies could be highly impactful. However, the resulting framework must be applicable in clinical settings and address data-related and underlying system-specific challenges to obtain effective results.

References
[1] Albers, D.J., Blancquart, P.-A., Levine, M.E., Seylabi, E.E., & Stuart, A. (2019). Ensemble Kalman methods with constraints. Inverse Prob., 35(9), 095007.
[2] Albers, D.J., Levine, M.E., Sirlanci, M., & Stuart, A.M. (2019). A simple modeling framework for prediction in the human glucose-insulin system. Preprint, arXiv:1910.14193.
[3] Albers, D.J., Levine, M.E., Stuart, A., Mamykina, L., Gluckman, B., & Hripcsak, G. (2018). Mechanistic machine learning: How data assimilation leverages physiologic knowledge using Bayesian inference to forecast the future, infer the present, and phenotype. J. Am. Med. Inform. Assoc., 25(10), 1392-1401.
[4] Arabi, Y.M., Dabbagh, O.C., Tamim, H.M., Al-Shimemeri, A.A., Memish, Z.A., Haddad, S.H., ... & Sakkijha, M.H. (2008). Intensive versus conventional insulin therapy: A randomized controlled trial in medical and surgical critically ill patients. Crit. Care Med., 36(12), 3190-3197.
[5] Brunkhorst, F.M., Engel, C., Bloos, F., Meier-Hellmann, A., Ragaller, M., Weiler, N., ... & Reinhart, K. (2008). Intensive insulin therapy and pentastarch resuscitation in severe sepsis. New Eng. J. Med., 358(2), 125-139.
[6] Clain, J., Ramar, K., & Surani, S.R. (2015). Glucose control in critical care. World J. Diabetes, 6(9), 1082-1091.
[7] Green, D.M., O’Phelan, K.H., Bassin, S.L., Chang, C.W.J., Stern, T.S., & Asai, S.M. (2010). Intensive versus conventional insulin therapy in critically ill neurologic patients. Neurocrit. Care, 13(3), 299-306.
[8] Herpe, T.V., Espinoza, M., Haverbeke, N., De Moor, B., & Van den Berghe, G. (2007). Glycemia prediction in critically ill patients using an adaptive modeling approach. J. Diabetes Sci. Technol., 1(3), 348-356.
[9] Hripcsak, G., & Albers, D.J. (2013). Next-generation phenotyping of electronic health records. J. Am. Med. Inform. Assoc., 20(1), 117-121.
[10] NICE-SUGAR Study Investigators. (2009). Intensive versus conventional glucose control in critically ill patients. New Eng. J. Med., 360(13), 1283-1297.
[11] Pritchard-Bell, A., Clermont, G., Knab, T.D., Maalouf, J., Vilkhovoy, M., & Parker, R.S. (2017). Modeling glucose and subcutaneous insulin dynamics in critical care. Control Eng. Pract., 58, 268-275.
[12] Suhaimi, F., Compte, A.L., Preiser, J.-C., Shaw, G.M., Massion, P., Radermecker, R., ... & Chase, J.G. (2010). What makes tight glycemic control tight? The impact of variability and nutrition in two clinical studies. J. Diabetes Sci. Technol., 4(2), 284-298.
[13] Van den Berghe, G., Wilmer, A., Hermans, G., Meersseman, W., Wouters, P.J., Milants, I., ... & Bouillon, R. (2006). Intensive insulin therapy in the medical ICU. New Eng. J. Med., 354(5), 449-461.
[14] Van den Berghe, G., Wouters, P., Weekers, F., Verwaest, C., Bruyninckx, F., Schetz, M., ... & Bouillon, R. (2001). Intensive insulin therapy in critically ill patients. New Eng. J. Med., 345(19), 1359-1367.

	Melike Sirlanci is an assistant professor in the Department of Pediatrics, Section of Informatics and Data Science at the University of Colorado’s Anschutz Medical Campus. She works on problems that arise in biomedicine by using both physiology-based mechanistic modeling and machine learning methods.
	David J. Albers is an informaticist and applied mathematician who develops mathematical machinery to model and understand clinically collected data. He is an associate professor in the Department of Pediatrics, Section of Informatics and Data Science; Department of Biomedical Engineering; and Department of Biostatistics and Informatics at the University of Colorado’s Anschutz Medical Campus. Albers is also an adjunct assistant professor in the Department of Biomedical Informatics at Columbia University Medical Center.
	George Hripcsak is a professor of biomedical informatics and chair of the Department of Biomedical Informatics at Columbia University. He is a board-certified internist with degrees in medicine and biostatistics who focuses on nonlinear time series analysis of electronic health record data and causal inference.