Replacing the "Giant Machine" of Weather Forecasting with a Single AI—Research That Performs the Work of 1000 Supercomputers in One Second

The process of weather forecasting is actually a much larger system than we imagine. It involves taking in vast amounts of data from satellites, buoys, balloons, and ground observation stations, estimating atmospheric conditions, solving fluid dynamics equations to predict the future, and finally translating it into regional forecasts—this entire process is processed by dedicated supercomputers over many hours. The research we'll introduce here, "Aardvark Weather," has managed to replace this entire massive machine-based process with just one machine learning model. Let's take a closer look at this paper published in the academic journal "Nature."

Why Has "Replacing the Entire Process" Been Avoided Until Now?

Many studies on AI-based weather forecasting have been published before. However, most of them only replaced "part of the forecasting pipeline" with machine learning. Specifically, while the AI ​​was entrusted with the "future prediction" portion after the initial atmospheric state was given, the most difficult process—"data assimilation"—which involves estimating the current atmospheric state from observational data, still relied on conventional numerical calculation systems.

Another 2021 paper cited by the authors of this paper stated that "several fundamental breakthroughs are needed to achieve this goal (complete end-to-end integration)." In other words, even among experts, there was a general feeling that this was "still many years away." The core of this report lies in the fact that a research team led by Richard Turner of Cambridge University has achieved this sooner than expected.

A Three-Stage Structure: Encoder, Processor, and Decoder

The internal structure of Aardvark Weather is simply organized. Three modules each have their own roles.

1. Encoder: Receives raw observational data from satellites, ground stations, ships, and balloons (radiosondes) to estimate the current atmospheric conditions. 2. Processor: Based on the estimated conditions, it sequentially generates forecasts from one day to ten days ahead, in 24-hour increments. 3. Decoder: Converts the obtained forecasts into local forecasts for specific locations (observatories).

These three modules are all composed of relatively lightweight neural networks (a combination of Vision Transformer and U-Net), and the total training time was approximately 100 hours using four NVIDIA A100 GPUs. Compared to the massive investment required to develop and maintain conventional numerical weather prediction systems—which involve huge dedicated teams and supercomputers—this is remarkably lightweight.

Accuracy Comparable to Conventional Models with Only 8% of Observational Data

What's particularly interesting from a technical standpoint is that Aardvark achieves accuracy equivalent to, or even exceeding, that of operational numerical weather prediction systems (such as the US GFS) across multiple variables and forecast periods, despite incorporating only approximately 8% of the observational data used by conventional operational systems.

Furthermore, for local forecasts at specific locations (temperature and wind speed), it demonstrates performance nearly on par with corrected forecasts in Europe and the United States for forecasts up to 10 days ahead. Particularly noteworthy is that Aardvark clearly outperforms existing corrected forecasts in regions lacking sufficient forecasting infrastructure, such as West Africa and the Pacific region. The fact that this technology benefits regions with limited forecasting infrastructure is significant from a practical value perspective.

An Overwhelming Speed ​​Difference: "1 Second" vs. "1000 Node Hours"

Another point emphasized in the paper is the dramatic difference in computational cost. Aardvark can generate a complete forecast from observational data in approximately 1 second using four GPUs. In contrast, major European numerical weather prediction systems (HRES) require approximately 1,000 node-hours (equivalent to 1,000 hours of computation using multiple computing nodes) just for data assimilation and forecast generation. This difference widens further when regional post-processing is included.

This speed difference is not merely boastful; it has practical implications. The ability to generate forecasts quickly and inexpensively opens up previously impractical applications, such as completely retuning the model to suit specific regions or purposes (agriculture, insurance, renewable energy, etc.). The paper actually reports that retuning the entire model end-to-end by narrowing down regions and variables improved temperature prediction errors by up to 6%. The authors modestly but strongly point out that this improvement is "comparable to the improvements achieved by the ECMWF (European Centre for Medium-Range Weather Forecasts) over more than a year with a large research team."

Limitations are also frankly described

The paper's discussion section carefully describes the limitations. Currently, Aardvark does not operate at the same spatial resolution as state-of-the-art operational systems (IFS), and it does not yet support "ensemble forecasts" (probabilistic forecasts incorporating uncertainty) that combine multiple forecasts. Furthermore, how to integrate data from new observation instruments for which training data does not yet exist is cited as a future challenge.

Nevertheless, the authors position Aardvark as "a pioneer of a new generation of end-to-end weather forecasting systems." The code and trained models are planned to be released on GitHub, which is a positive point as it facilitates replication and further research.

Thoughts from a Researcher

Replacing a massive engineering system like weather forecasting, which has been refined over decades, with a single, relatively lightweight model—the audacity of this idea, and the execution that brought it to a level that passed peer review in Nature, are both what make this paper so compelling. It's important to know that behind the flashy headline, "AI Predicts the Weather," lies a painstaking technical challenge that involved even the most difficult process of data assimilation.