Anemoi: A new framework for weather forecasting based on machine learning

October 17th, 2024

This is a six-day forecast produced by ECMWF’s experimental Artificial Intelligence Forecasting System (AIFS) from 06 UTC on 3 October 2024. It shows temperature at 850 hPa (colors, in °C) and 500 hPa geopotential (lines, in decameters). Credit: European Center for Medium-Range Weather Forecasts (ECMWF)

A range of national meteorological services across Europe and ECMWF have launched Anemoi, a framework for creating machine learning (ML) weather forecasting systems. Named after the Greek gods of the winds, Anemoi is a collaborative, open-source initiative.

The goal of Anemoi is to provide key building blocks to train state‑of-the‑art data-driven models and to run them in an operational context. As a framework, it seeks to enable meteorological organizations to train machine-learning models with their own data.

Anemoi pools expertise and resources at a time of rapid changes in weather forecasting: Machine learning models have entered the scene, in addition to traditional physics-based models.

At ECMWF, we have already developed the experimental Artificial Intelligence Forecasting System. Anemoi builds on top of these developments, generalizing what started as an AIFS codebase to allow wider functionality for a greater set of users.

The AIFS is a global system, but work has also been going on in Anemoi to create regional (limited-area) ML systems. For example, MET Norway has created a regional model for Scandinavia (Nipen et al., 2024, arXiv:2409.02891).

And the German National Meteorological Service (DWD) is developing a data-driven weather forecasting model based on data from its ICOsahedral Non-hydrostatic (ICON) model, called Artificial intelligence ICON (AICON).

Anemoi is not just a technical framework but a philosophy of open collaboration. The code is freely available on GitHub, under a permissive license, meaning that anyone can contribute to its development or use it for their activities.

"Anemoi has the potential to democratize access to and accelerate further development of data-driven weather forecasts," says ECMWF's Deputy Director-General, Florian Pappenberger.

In January 2024, a new version of the AIFS with a grid spacing of 28 km was introduced, down from 111 km. This improved forecast scores, as shown here for 500 hPa geopotential in the northern hemisphere for 2022 (higher means better). The other lines show the performance of ECMWF's Integrated Forecasting System (IFS) and of the machine learning systems Google DeepMind's GraphCast and Huawei's Pangu-Weather.

The plot shows a 7-day forecast of 10 m wind speed (shading) and sea-level pressure (contours). The model has learned to forecast at high resolution (here ~10 km) inside the Nordic region, and at low resolution (here ~100 km) outside of this domain. The model successfully creates a higher-resolution structure over the Nordics. Credit: European Center for Medium-Range Weather Forecasts (ECMWF)

Components

Anemoi comprises several packages written in the Python programming language. These packages address different aspects of the artificial intelligence (AI) weather forecasting pipeline.

Documentation on them has been brought together in a series of web pages:

Anemoi Datasets: This component generates ML–optimized datasets from various sources and data formats of meteorological data and observations, complete with rich metadata, for example, an optimized subset of ECMWF's ERA5 reanalysis or of a historical operational analysis dataset. It streamlines the often cumbersome process of data preparation, ensuring that high-quality, consistent, and optimized data is available for model training.

Anemoi Training: Understanding that flexibility is key in ML model development, this module allows users to modify most parts of the training through configuration files, without needing to alter the underlying code. This approach democratizes access, enabling meteorologists without deep coding expertise to experiment with advanced data-driven weather prediction models.

Anemoi Models: This package houses the model code, designed with efficiency and minimal dependencies in mind. It ensures that the transition from development to deployment is as smooth as possible.

Anemoi Inference: Building on ECMWF's experience with the AI Models tool, which has been running daily experimental ML forecasts since 2023, Anemoi Inference enables fast operational deployment of trained models. This component is crucial for integrating ML forecasts into time-sensitive operational workflows.

Anemoi Graphs: Supporting custom graph generation, this module is particularly exciting for researchers exploring novel graph architectures. It already supports multi-scale GraphCast-like graphs and stretched-grid graphs showcased recently by MET Norway, with more innovations on the horizon. Graphs can be easily visualized to understand the connectivity.

Current involvement

Anemoi currently involves the Spanish State Meteorological Agency (AEMET), the Danish Meteorological Institute (DMI), the German National Meteorological Service (DWD), the Finnish Meteorological Institute (FMI), the Italian Air Force Meteorological Service (ITAF Met Service), the Royal Netherlands Meteorological Institute (KNMI), MET Norway, Météo-France, MeteoSwiss, Belgium's Royal Meteorological Institute (RMI) and ECMWF.

"These organizations and others can make more contributions to Anemoi, which is intended as a tool by the community and for the community," says Florian.

"There will be room in the Anemoi community for a wide range of people, from ML practitioners and meteorologists to students looking to shape the future of weather forecasting."

Provided by European Centre for Medium-Range Weather Forecasts (ECMWF)