Machine Learning: Dynamical Tests of AI-Driven Weather Models

Our working group focuses on the dynamical testing of several cutting-edge, AI-based global weather prediction models. Specifically, we are evaluating the performance of NVIDIA’s SFNO, Google DeepMind’s GraphCast, and Huawei Cloud’s Pangu-Weather under shared/similar experimental protocols. Inference for these models is conducted using a modified version of NVIDIA’s Earth-2MIP framework, incorporating unique initial conditions derived from our test suite.

NVIDIA SFNO (Spectral Fourier Neural Operator)

The SFNO model is a Fourier-based neural operator model developed as part of NVIDIA’s Earth-2 efforts. The 73-channel version ingests a wide array of input variables, including:

• Atmospheric fields at multiple pressure levels (e.g., temperature, geopotential height, u/v wind)

• Surface variables (e.g., 2m temperature, mean sea-level pressure, soil moisture)

The model operates in spectral space and provides high-resolution forecasts via Fourier transformations.

• Further Info: https://arxiv.org/abs/2311.16372
  
  

DeepMind GraphCast (Small Framework)

GraphCast is a spatiotemporal graph neural network that predicts global weather fields based on a graph representation of the Earth’s surface. The "small" version of GraphCast (1x1-degree, 83-channel) used in our test suite includes:

• Core meteorological variables on a latitude-longitude grid mapped to graph nodes (e.g., temperature, wind components, pressure)

• Historical lag inputs to model timestepping (two in, one out)

GraphCast achieves state-of-the-art performance with high efficiency by leveraging message passing between spatial nodes over time.

• Further Info: https://www.nature.com/articles/s41586-023-06557-7
  
  

Huawei Pangu-Weather

Pangu-Weather is a 3D transformer-based model developed by Huawei Cloud. It predicts atmospheric fields at multiple isobaric levels with fine vertical resolution and offers fast inference at high spatial resolutions. Input features include:

• Full 3D fields (temperature, geopotential height, wind) across pressure levels

• Surface variables and static fields

• Pangu-Weather demonstrates competitive skill, particularly at medium-range forecasts (3-7 days), and emphasizes model stability over time.

• Further Info: https://www.nature.com/articles/s41586-023-06034-3
  
  

Earth-2MIP Framework

NVIDIA’s Earth-2MIP (Model Intercomparison Platform) provides a standardized environment for running and evaluating AI-based weather models. It supports consistent input/output handling, interpolation, and regridding, and integrates evaluation metrics in line with physical consistency and accuracy.

Our working group employs a modified version of Earth-2MIP, enabling:

• Customized initial conditions from ECMWF reanalyses (ERA5)

• Unified inference pipelines for SFNO, GraphCast, and Pangu-Weather

• Streamlined comparisons under identical(similar) dynamical conditions

• Further Info: https://github.com/taobrienlbl/DCMIP2025-ML (original: https://github.com/NVIDIA/earth2mip)