Custom software increases speed, ensuring high-quality ocean sampling

At the beach, ocean waves provide soothing white noise. But in scientific laboratories, they play an important role in weather forecasting and climate research. Along with the atmosphere, the ocean is usually one of the largest and most computationally demanding components Earth system models Like the Energy Department’s Energy Exascale Earth System Model, or E3SM.

Most modern ocean models focus on two types of waves: a barotropic system with fast wave propagation speeds and a baroclinic system with slow wave propagation speeds. To address the challenge of simulating these two methods simultaneously, a team from DOE’s Oak Ridge, Los Alamos, and Sandia National Laboratories has developed a new solution algorithm that reduces the total runtime of the model. Ocean, E3SM’s ocean circulation model, 45%.

The researchers tested their software on the Summit supercomputer at ORNL’s Oak Ridge Leadership Computing Facility, a DOE of Science user facility, and the Compy supercomputer at the Pacific Northwest National Laboratory. They ran their primary simulations on the Corey and Perlmutter supercomputers at Lawrence Berkeley National Laboratory’s National Energy Research Science Computing Center. Results published In International Journal of High Performance Computing Applications.

Because TrilinosA database of open-source software optimized for solving scientific problems on supercomputers, written in the C++ programming language, and Earth system models such as E3SM typically written in Fortran, the team used a related software library called ForTrilinos that includes Fortran interfaces. In existing C++ packages, it focuses on barotropic waves to design and customize a new solution.

„One useful feature of this interface is that every component of the C++ package can be used in the Fotron language, so there is no need to translate anything, which is very convenient,” said lead author Hyun Kang. ORNL.

This work is in progress The results of the study were published In earlier times Journal of Advances in Modeling Earth Systems Researchers at ORNL and Los Alamos National Laboratory developed an index by hand to improve MPAS-Ocean.

Now, the ForTrilinos-implemented solver has overcome the remaining shortcomings of the solver from the previous study, especially when users run MPAS-Ocean using a limited number of compute cores for a given problem size.

MPAS-Ocean’s default solver relies on apparent sub-circulation, a technique that uses many small time intervals or time steps to calculate the properties of barotropic waves in conjunction with barotropic calculations without perturbing the model.

If a baroclinic wave and a barotropic wave can advance with time step sizes of 300 seconds and 15 seconds, respectively, the barotropic calculation must complete 20 times more iterations to maintain the same speed, which takes a large amount of computing power.

In contrast, the new solution for the barotropic system is semi-implicit, meaning it is unconditionally stable, allowing researchers to save significant amounts of time and computing power without sacrificing accuracy for the same number of large time steps.

A community of software developers has spent years developing various climate applications on TRILINOS and FORTRILINOS, so the latest MPAS-Ocean solution that leverages this resource outperforms a hand-crafted solution, allowing other scientists to accelerate their climate research efforts.

„If each algorithm had to be coded individually, it would require a lot of effort and expertise,” Kang said. „But with this software, we can immediately run simulations at faster speeds by incorporating optimized algorithms into our program.”

Although the current solver still has scalability limitations on high-performance computing systems, it performs well up to a limited number of processors. This drawback exists because all processors must communicate at least 10 times per time step, which slows down the performance of the model. To overcome this obstacle, researchers are currently improving processor communication and sending the solution to GPUs.

In addition, the team updated the time-stepping method for the baroclinic system to further improve the performance of MPAS-Ocean. With these advances, researchers aim to make climate forecasts faster, more reliable, and more accurate, which are essential improvements to ensure climate security, enable timely decision-making, and enable high-resolution predictions.

„This barotropic mode solver enables fast computation and consistent integration of models, especially MPAS-Ocean,” Kang said. „Extensive use of computing resources requires large amounts of electricity and energy, but by accelerating this model we can reduce that energy use, improve simulations, and more easily predict the effects of climate change decades or thousands of years into the future.”