Massive supercomputer simulation models universe from near birth until today

One of the largest cosmological simulations ever run
November 2, 2015

Galaxies have halos surrounding them, which may be composed of both dark and regular matter. This image shows a substructure within a halo in the Q Continuum simulation, with “subhalos” marked in different colors. (credit: Heitmann et al.)

The Q Continuum simulation, one of the largest cosmological simulations ever performed, has modeled the evolution of the universe from just 50 million years after the Big Bang to the present day.

DOE’s Argonne National Laborator led the simulation on the Titan supercomputer at DOE’s Oak Ridge National Laboratory.

Over the course of 13.8 billion years, the matter in the universe clumped together to form galaxies, stars, and planets. These kinds of simulations help scientists understand dark energy (a form of energy that affects the expansion rate of the universe) and the distribution of galaxies, composed of ordinary matter and mysterious dark matter.

This series run on the Titan supercomputer simulates the evolution of the universe. The images give an impression of the detail in the matter distribution in the simulation. At first, the matter is very uniform, but over time, gravity acts on the dark matter, which begins to clump more and more, and in the clumps, galaxies form. (credit: Heitmann et. al.)

Intensive sky surveys with powerful telescopes, like the Sloan Digital Sky Survey and the new, more detailed Dark Energy Survey, show scientists where galaxies and stars were when their light was first emitted. And surveys of the Cosmic Microwave Background (light remaining from when the universe was only 300,000 years old) show us how the universe began — “very uniform, with matter clumping together over time,” said Katrin Heitmann, an Argonne physicist who led the simulation.

The simulation fills in the temporal gap to show how the universe might have evolved in between: “Gravity acts on the dark matter, which begins to clump more and more, and in the clumps, galaxies form,” said Heitmann.

The Q Continuum simulation involved half a trillion particles — dividing the universe up into cubes with sides 100,000 kilometers long. This makes it one of the largest cosmology simulations at such high resolution. It ran using more than 90 percent of the supercomputer.

“This is a very rich simulation,” Heitmann said. “We can use this data to look at why galaxies clump this way, as well as the fundamental physics of structure formation itself.”

Analysis has already begun on the two and a half petabytes of data that were generated, and will continue for several years, she said. Scientists can pull information on such astrophysical phenomena as strong lensing, weak lensing shear, cluster lensing, and galaxy-galaxy lensing.


Abstract of The Q Continuum simulation: Harnessing the power of GPU accelerated supercomputers

Modeling large-scale sky survey observations is a key driver for the continuing development of high-resolution, large-volume, cosmological simulations. We report the first results from the “Q Continuum” cosmological N-body simulation run carried out on the GPU-accelerated supercomputer Titan. The simulation encompasses a volume of (1300 Mpc)3 and evolves more than half a trillion particles, leading to a particle mass resolution of ${m}_{{\rm{p}}}\simeq 1.5\cdot {10}^{8}\;$ ${M}_{\odot }$. At this mass resolution, the Q Continuum run is currently the largest cosmology simulation available. It enables the construction of detailed synthetic sky catalogs, encompassing different modeling methodologies, including semi-analytic modeling and sub-halo abundance matching in a large, cosmological volume. Here we describe the simulation and outputs in detail and present first results for a range of cosmological statistics, such as mass power spectra, halo mass functions, and halo mass-concentration relations for different epochs. We also provide details on challenges connected to running a simulation on almost 90% of Titan, one of the fastest supercomputers in the world, including our usage of Titan’s GPU accelerators.