Traditional weather forecasting relies on a combination of weather observations and data models. Meteorologists produce weather forecasts by gathering as much data as possible and then processing it through weather prediction models. Government-backed organisations, such as the National Weather Service (NWS) in the United States and the European Centre for Medium-Range Weather Forecasts (ECMWF), typically create these models. Meteorologists develop forecasts based on models from weather agencies, as well as models created by private weather forecasters.
Weather information for the prediction models comes from surface observations provided by thousands of automated weather stations around the world, as well as from radars and satellites. Surface observation includes atmospheric data such as temperature, wind speed, humidity, and precipitation. Meteorologists then put the observed data into weather models to create weather forecasts. Thanks to the proliferation of computers to process these models, weather forecasts are now designed with much higher accuracy than they were a century ago. Advancements in computing power, such as the use of supercomputers, means that weather forecasts have continued to become even more accurate.
Making the most accurate weather predictions, particularly regarding precipitation, requires more than surface observations. This is where radar and satellite come into the picture. Weather radars work by transmitting radio waves into the atmosphere, and the waves bounce off of objects like rain and snow, thereby telling radars the exact location of precipitation.
Radars are highly accurate in determining current precipitation, and information from radars can be combined with surface observations to make short term precipitation forecasts. However, there are limitations to radar. They can detect precipitation only in a certain radius from the station, and comprehensive radar coverage tends to be limited to highly developed countries. For example, while the United States has 160 radars covering all populated areas, Russia, a country roughly 1.7 times the size of the U.S., has around 30 radar stations.
Weather satellites help overcome the limitations of radars. Satellites observe precipitation over a wide area, but can also detect visible and infrared atmospheric readings beyond what’s possible with surface data. Satellites are particularly useful for observing precipitation outside the range of weather stations; for example, the weather images that blanket the news during oceanic events like tropical cyclones typically come from meteorological satellites. The roughly 80 weather satellites above the world are either polar orbiting, meaning they orbit the Earth by crossing both poles, or geostationary, in which the satellites orbit in sync with the Earth, continually observing the same area.
Intelligent weather predictions
Technological advancements in the 21st century have brought many improvements to weather forecasting. The growth of smartphones has brought on-the-go weather forecasting to billions of people around the world, while the location data of the devices improves the accuracy of forecasting. Another recent development, the AI revolution, has not spared weather prediction either. Developments in machine learning mean that AI can be incorporated into existing weather models to produce even more accurate forecasts. Machine learning models for weather forecasting quickly process large amounts of weather data, and they can compare data from weather stations and satellites with traditional forecasts to make highly accurate predictions.
Machine learning for more accurate forecasting
One of the main benefits of introducing machine learning to weather forecasting is more accurate predictions. Machine learning can be used to process immediate comparisons between historical weather forecasts and observations. With the use of machine learning, weather models can better account for prediction inaccuracies, such as overestimated rainfall, and produce more accurate predictions.
At Yandex.Weather, we use machine learning to provide highly accurate weather forecasts. Our prediction model uses gradient boosting, a machine learning technique for building predictive models, to correct for any errors that come from traditional weather forecasting. We use the open-source library, CatBoost, and train our model to compare data from weather stations to past weather reports. The trained model can filter out any errors and build accurate forecasts based on current weather conditions.
Expanding nowcasting with deep learning
Aside from more accurate forecasts, machine learning can also be used to improve nowcasting, which is immediate weather prediction, typically within two hours, that provides minute-by-minute precipitation forecasts. While nowcasting is technically possible through traditional forecasting using radar data, weather models based on machine learning can also take into account data from weather satellites. Integrating machine learning into weather models enables them to quickly process satellite images for nowcasting. Adding weather satellites to the tech behind nowcasting greatly expands its reach. With machine learning, potentially anyone in range of a weather satellite can use nowcasting, rather than just those living near a radar station.
We’ve used deep learning in particular to improve Yandex.Weather’s nowcasting. Deep learning is an AI technology that aims to replicate how humans process things like images. In weather forecasting, prediction models use deep learning primarily to process images from weather satellites. With our deep learning network, we’ve expanded the reach of our nowcasting beyond radar stations. Our users are mostly in Russia, where radar coverage is limited to populated areas of European Russia. By using deep learning to add satellite images to our weather model, we've expanded nowcasting to the areas covered by the two satellites we're using. Practically all of our users in Russia now have access to nowcasting, along with anyone else living under the satellites.
Aside from the introduction of AI, weather forecasting has changed with another recent technological innovation; the smartphone. People with smartphones can access detailed weather reports wherever they are, and these forecasts are more accurate thanks to their devices. Weather predictions can now take into account the specific location data from smartphones to provide users with hyperlocal forecasting. Since a city’s particular weather can vary dramatically from one block to the next, the use of location data from smartphones has significantly advanced forecasting for each user and exact locations.
The future of weather forecasting
The last few decades have been transformative for the advancement of weather forecasting. Looking ahead, weather modelling stands to grow even more accurate for a greater number of people around the world.
As machine learning advances and more weather models start integrating it, weather forecasting will become increasingly accurate. There is also excellent potential for a global expansion of nowcasting, a relatively recent addition to consumer weather forecasting. Only a select few weather services include nowcasting in their forecasts, and, in the past, the tech was limited to people in areas with reliable radar coverage. As we’ve shown at Yandex.Weather, machine learning can be added to weather forecasting to extend nowcasting to places such as Russia that lack widespread radar coverage. As smartphone penetration grows worldwide, particularly in the same areas lacking weather radars, more people will gain access to accurate, hyperlocal weather forecasting as well.
Alex Ganshin, Head of Meteorology, Yandex