Modeling the Universe with Deep Learning

By Jillian Kunze

Around the year 1900, scientists thought the universe was made up entirely of baryonic matter — ordinary matter consisting of protons, neutrons, and electrons. But by 1970, researchers were starting to think that only 15% of the universe was made up of baryonic matter, and the rest was composed of dark matter. And now, scientists believe that the universe is made up of 27% dark matter, 68% dark energy, and only 5% baryonic matter (though these numbers change a little based on what experiment they are being derived from). “Dark energy is a term that characterizes our lack of understanding of the universe” said Shirley Ho of Carnegie Mellon University and the Flatiron Institute. Though dark energy is necessary to explain certain observations of the universe, most of its qualities are not at all well understood. 

Ho believes that the nature of dark matter and energy are the most interest questions in physics today; unfortunately, they are also the most expensive. Telescopes or experimental facilities to search for further evidence of dark matter or energy can cost billions of dollars. But these expensive large-scale endeavors are not the only way to investigate the mysteries of space. During a minisymposium presentation at the 2021 SIAM Conference on Computational Science and Engineering, which is taking place virtually this week, Ho discussed work she completed with previous graduate students to create the first deep learning simulation of the universe. They began the project in 2016, when the field of deep learning was still relatively new. 

Modeling the universe is a complex and computationally demanding problem, but deep learning approaches can help. The architecture in Ho’s project was somewhat related to image recognition, which has rapidly improved in recent years through deep learning approaches. The input for the model was an analytical approximation of the universe’s current structure. But instead of using Newton’s laws to numerically simulate how the particles that make up this structure would move, they instead attempted to use deep learning based on a large number of pre-run simulations to learn—or interpolate—what the particles would do. The output was then a numerically simulated field of the positions and velocities of these particles as they would evolve based on gravitational attraction. The training set of the deep learning model was made up of 8,000 pairs of three-dimensional boxes containing simulations of the structure of the universe. These simulations were based off of standard cosmological parameters, whose values are generally agreed upon by most astronomers. The machine learning model was based in the U-Net architecture, which is a slight variant upon residual neural networks.

To determine the accuracy of the deep-learning-generated simulations, Ho and her collaborators compared the output of their model to a simpler benchmark model that is commonly used by researchers in the field to produce simulations relatively quickly. They specifically compared the two models in terms of the displacement field, or the difference between the current position and initial position of the particles. The benchmark model took around 15 to 20 minutes to run, and had a fairly large maximum error. The machine learning model, on the other hand, had 10 times less error; Ho now believes that her team could improve upon it even more.

The researchers compared the average power spectra produced by 1,000 simulations of the machine learning and benchmark models to the true power spectrum. They also computed the ratios of the power spectra that were output by the benchmark model and the machine learning model to the true power spectrum, as well as compared the relevant cross-correlation coefficients. The machine learning model beat the benchmark model significantly, as revealed by how much closer it matched the true power spectrum when plotted side-by-side. It also was able to create the power spectrum on a significantly shorter timescale — around 60 million times faster than the usual simulation.

Ho next asked whether the model could extrapolate instead of just interpolate data. As the training model was based on a universe with just one particular amount of dark matter, she tested the machine learning approach by varying the input of the amount of dark matter in the universe and investigating whether the model would still do well. Once again, the machine learning model performed better than the benchmark model on tests with several different amounts of dark matter.

At the end of her talk, Ho noted that her team did not need to overlap the training set with the test set or explicitly use any transfer learning or meta-learning for their machine learning model. She pondered what this meant in grander terms for our understanding of the universe. “Maybe there is overlap in information somewhere between these modeled universes?” said Ho. “Or maybe the universe is fairly simple, so that the generalization and extrapolation by network is relatively easy?” She concluded her presentation with some provocative final questions: what role deep learning will play as an approximate simulator for physical processes, and to what level it will be possible to understand deep learning and how and why it extrapolates information. Ultimately, Ho hopes to use machine learning to further scientific knowledge about the physics of the cosmos.

Jillian Kunze is the associate editor of SIAM News.