Abstract:
Stars spend their lives generating energy via thermonuclear fusion. Their overall evolution is driven by the changing composition in their interiors, and at the late stages of evolution and during explosive events, the nuclear timescale can be short and drive strong hydrodynamic flows. Modeling these environments is challenging. Our group has been developing simulation codes that can accurately model stellar reactive flows, through novel time-integration methods that strongly couple reactions and hydrodynamics. Furthermore, our codes are designed to run on modern GPU-based supercomputers. In this talk I'll describe some of the algorithmic work we have done for stellar reacting flows and show some applications to different types of supernovae. This work is done in our open source AMReX-Astrophysics simulation suite (https://github.com/amrex-astro/).
Bio:
Michael Zingale is a Professor of Physics and Astronomy at Stony Brook University. He earned a BS in Physics and Astronomy (1996) from the University of Rochester and a PhD in Astronomy and Astrophysics (2000) from the University of Chicago. He was part of the Flash Code development team that won a Gordon Bell Prize in 2000, and received a Presidential Early Career Award for Scientists and Engineers (PECASE) through DOE NNSA in 2005, and an Outstanding Junior Investigator award for the DOE Office of Nuclear Physics in 2006. Michael's research involves the development of new algorithms for efficiently modeling convection in stellar interiors. He is a codeveloper of the Open Source astrophysical, multiscale/multiphysics simulation codes Castro and MAESTROeX and the nuclear astrophysics library pynucastro. He applies these codes to studies of early phases of Type Ia supernovae, novae, and X-ray bursts.
Summary
Focus: efficient simulation of stellar dynamics
Stars 101:
Fuse Hydrogen to Helium
Process is very Temperature sensitive
Internal Pressure/Temperature pushing against Gravitational collapse
More massive stars are hotter, fuse faster
Usually in thermal equilibrium
Once Hydrogen is used up, fuse Helium into heavier elements until Iron is reached
Iron can’t be fused by stars so that’s the end of a star’s life
Low mass stars end their lives as white dwarfs
High mass stars reach Iron, collapse and undergo collapse, then supernova
Most stars are in binary systems and if they interact and transfer mass they can re-energize and ultimately explode in other ways
Simulation challenges:
Physical processes”
Hydrodynamics of moving gas (conservation of mass, momentum, energy)
Combustion / nuclear reactions (similar to chemical reaction)
Self-gravity
Radiation/diffusion
Magnetic fields
Mostly modeled in 1D
However, convection and binary interactions, magnetic fields, etc. are 3D
Modeling conservation laws:
Discretize star in space on a grid (regular or adaptively refined)
Update state of each cell based on its neighbor
Complexity is computing mass/energy fluxes through cell boundaries
Mesh parallelized onto a parallel computer
Time step:
Mesh resolution constrained by the speed at which information is transmitted
Must be fine enough that information can’t propagate through more than one mesh cell in one time step
There are some techniques to get around this…
More complex with multi-physics
E.g. Reaction + hydrodynamics
Reactions change composition + inject energy
Change the material that is moving
This can destabilize the simulation because the reaction model and hydrodynamics push the simulation in different directions, so they must be kept tightly coupled
Approach: modify the time step based on the rate of the reactions
AMReX astrophysics suite: https://github.com/amrex-astro
MAESTROeX: https://amrex-astro.github.io/MAESTROeX: massively parallel low Mach number stellar hydrodynamics code
Castro: https://amrex-astro.github.io/Castro: adaptive mesh, astrophysical radiation/MHD/hydrodynamics simulation code
Nyx: https://amrex-astro.github.io/Nyx: adaptive mesh, cosmological hydrodynamics simulation code
Pynucastro: https://github.com/pynucastro/pynucastro: framework for working with reaction databases
Outputs code that simulates known reactions
Computers
OLCF Frontier
9408 nodes
1 AMD CPU (64 cores)
4 AMD GPUs (x2 dies)
1 exaflop
Conversion of Castro to GPUs started in 2018
Moved from Fortran to C++ (preferred by GPU software stacks) since 2020
Move all data to GPU and do all computation there
CPU not used except for IO
Achieves good parallelism on large clusters
Approaches for coupling multi-physics
Operator (Strang) Splitting:
Alternate reactions and hydro steps
Model reactions with old hydro for half time step
Then update hydro based on that
Then model reaction on updated hydro
Simplified-SDC
Model hydro advections using reactions from last time step
Then recompute the hydro and reactions based on just-computed advection and current time step reaction
This does more work per time step but with larger time steps, so less work overall
Example: Type Ia supernovae
No H, strong Si, Ca, Fe lines
Very rare, very dim beforehand, brighter than a whole galaxy afterwards
Consensus: thermonuclear explosion of carbon/oxygen white dwarf
Don’t know what type of white dwarf configuration is exploding
Modeling one scenario:
Small white dwarf pulling He from a companion
He forms a layer, gets very hot, explodes around the star
Creates a compression wave from outside in, compresses the C inside, causing it to burn, exploding it
Challenge: shock wave fronts are supersonic, normally require small time steps
Evaluation:
Simplified SDC reproduces the solution from operator splitting with much larger time step, less compute time
Now running 3D model of binary system
Example: massive stars
Burn through H -> He -> O ->....-> Iron core that can’t be fused
Iron core needs to support to pressure of star’s mass but without the energy output from fusion
This pressure can be resisted by electrons but only up to a point
At the high temperature inside the iron core,
High energy photons inside the core can break up nuclei into protons, neutrons and neutrinos
This leaks out neutrinos, making the core cooler
This process is often omitted in simulations
Improved operator coupling approach allows simulation of this process, including the core dynamics, without shrinking the time step too much
Resources
Pyro: hydro by example
Hydro code design with clarify to teach students about simulation techniques
2D solvers for major physical dynamics
Open astrophysics bookshelf: https://github.com/Open-Astrophysics-Bookshelf
Virtual Astronomy Software talks: https://vast-seminars.github.io