Forecasting the weather: Supercomputer or AI?

Weather forecasting Artificial Intelligence (AI) programmes have been developed by the major tech giants, including Google DeepMind’s GraphCast and Huawei’s Pangu-Weather system. They have already shown that instead of hours of expensive and environmentally-unfriendly supercomputer time running millions of lines of code, AI offers much better performance using a cheaper computer. In just a few minutes it can produce a 10-day weather forecast that is as good - sometimes better - than the world’s top weather forecasters with their supercomputers. Does this mark a turning point in the science of weather forecasting?

We rely on accurate weather forecasts and slowly, decade by decade, they are getting better. Today, the 6-day forecast is about as good as a 3-day forecast was in the 1990s. Conventional weather models divide the atmosphere into millions of cells to which are applied the laws of fluid dynamics, thermodynamics etc taking several hours using over a million processors to output just four updates per day, and each year forecasters want bigger and faster computers.

But is that progression nearing its end as AI begins to revolutionise weather prediction? Soon we might not need a supercomputer at all as we could be obtaining better results, quicker and at far less cost, on a humble laptop.

AI weather forecasting suites look for patterns in the way the atmosphere changes using a multi-decade database of actual weather observations combined with details of their analysis. In comparisons between supercomputers and AI programs, after being initialised with the same data, GraphCast outperformed the European Centre For Medium-Range Weather Forecasting (ECMRF) forecast out to 10 days for 90% of the time in less than a minute running on a desktop computer! The Pangu AI program offers a coarser view of the atmosphere than GraphCast but in trials even it produced forecasts showing the same skill of the ECMWF’s model. Currently AI scientists are increasing their efforts to use only observational data.

The ECMWF, based near Reading, England has already generated its own AI forecasts. It says that initially it could enable more frequent forecasts and free-up computer time for other problems. But does AI’s success run deeper?

It's too early, at least not this year, to predict that traditional weather forecasts will disappear but it does seem on the horizon, particularly during a time of economic difficulties. It could have a severe impact on the financial justification for future supercomputers as well as the current business model for some national weather agencies. Why pay a national agency for a weather forecast when you can do it just as well using an app?

If AI is about to conquer weather forecasting it is also going to take on it’s long-term cousin - climate prediction. The 40-year training datasets used to prime AI weather suites are too short for the same approach to be used for climate studies, but there is still a lot of longer-term data available and the ever adaptable AI will find a way to incorporate and use them.

Useful Limits

Last year I suggested that outside of their academic fascination, looked at in terms of their contribution to climate policy, supercomputers may have reached their useful limit. Also last year the Royal Society issued a report by Dame Julia Slingo, formerly of the UK Met Office, making the case for more computing power to produce better climate models saying that they are essential for “robust decision-making” in the future, and calling for a major new international facility modelled on CERN that would push climate modelling forward, with major new computing technology to be able to predict the weather and climate at kilometre-scale resolution resulting in better global climate predictions and services.”

Perhaps though we have already started building the facilities needed to take weather forecasting and climate modelling forward. They are the new AI startups, both independent and those associated with the tech giants?

Ultimately the decision will be made on results. Physical models run in supercomputers or AI programs on desktops? The answer may soon become obvious surprising some weather scientists weaned on supercomputers.

Feedback: david.whitehouse@netzerowatch.com