Ask HN: Why are we still so bad a weather forecasting?

My mother is now retired, but she used to work in the National Wheatear Forecast Office in Argentina. She says that now the weather forecasting is very accurate.

When she started working there, they didn't have satellite images, only many local measurement stations in each big city and other places. In each one, someone measure all the things manually and then send a message to the central office.[1] They have to interpolate manually the data to cover all the country.

The idea is that if you have some rain clouds here, and the wind has this speed and direction, then tomorrow they will be approximately there. The clouds may run out of water. Sometimes clouds appear out of the blue if hot humid air collide with cold air.

If the clouds were moving over the sea and they now move over land, you must make some corrections. If there are some mountains, they may change the speed or direction of the clouds. In particular the Andes may stop the clouds for a few days and the change of height make them drop most of the rain in Chile, so it's difficult to predict manually. I'm not a meteorologist, so I don't know the details, but you can see that this involve a lot of general and local knowledge, approximations and guessing.

She was involved in getting the first satellite images here, and then interpreting them, because they only know how the clouds looked from below. [2] Much later they made a transition to digital forecasting. I still remember a few maps in ASCII code with the border of the country, and the predictions of the pressure, humidity and other stuff. She was not involved too much in the modeling part, but she was using the result to make the official forecast.

The first models were not very good, and sometimes they have to use the manual predictions. In particular the mountains in the Andes were not correctly modeled, and we have a big ocean in each side to make everything more complicated.

After a few/many years the model become better and better. Also, they have satellites that can measure visible light, infrared, IIRC humidity and a lot of other stuff. Computer models use more variables, smaller grids, more accurate mountains and have better formulas. So the models now are pretty good.

She now is retired, but she still makes unofficial forecast for the family. If I'm planning a pick nick for the weekend, she will give (unsolicited) advice. [3]

[1] Sometimes the data was inaccurate. Sometimes the data was missing. Sometimes the local person was lazy and lied instead of going outside to get the data. Sometimes the problem was in the transcription.

[2] There is some standard classification of clouds and you must understand how each one looks form above. For example dark clouds are not dark from above, so the darkness is not a good indication of how much water they have.

[3] A few years ago (after she retired), we made the birthday party of my daughter in Zoo a few miles away of Buenos Aires. Two days before the party she said something like:

> It will be rainy in the morning, but after 10am it will be fine, so don't cancel the party. Also, it will rain again after 6pm, so remember to return before that time.

It was pretty accurate, like half an hour of error in the time. She was seeing the forecast of the National Office and also the forecast of a few international pages. Some of them publish the easy forecast in words, and also the maps with the predictions of the models.

I don't think we are bad at it. I can look at multiple weather sites, get hourly predictions that typically are correct, and even accurately predict the differences between my location and towns 30 minutes away. I can look at predicated radar for the next 4 hours, and accurately dodge storms when exploring the mountains. In my experience, forecasts are accurate all but a few days a year. To be fair, those few days tend to be notable... but if the forecasts are correct 360 days a year, with 5 glaring days of error, that still feels pretty good to me.

I had to look back, because I remembered a discussion here on HN based on an Atlantic article that took exactly the opposite point of view:

https://news.ycombinator.com/item?id=19045250

I'd have to disagree that we are bad at weather forecasting. While we are surprised every so often, the progress in accuracy of the modern five day forecast has been shocking over the arc of my 30-or-so years on the planet. We almost always have a substantial heads up about weather that is going to affect our movements around the world.

I agree with you that down to the minute precipitation forecasts aren't perfect (though I'd say the alerts I get from Dark Sky are at least pretty good) but that level of fidelity is asking a lot.

Here in Switzerland, there are tons and tons of measure points, and in my experience, the resulting predictions are fantastically accurate.

For example, here is one of the measurement stations in Zurich:

http://wetterstationen.meteomedia.de/messnetz/vorhersagegraf...

This is outside of the city center (to minimize effects such as radiant heat due to pavements etc.), so it gives a good base value to compare against.

Perhaps you've heard of the "Butterfly Effect" but not a formal definition of the term.

For the issue of predicting weather, the model of weather is "sensitive". The observation metaphor of the "butterfly effect" is that relatively small perturbations to the system have the observed effect of being indistinguishable between relatively large perturbations. We may substitute "small" with "deterministic" and "large" with "non-deterministic".

A summary of this statement would be that initial or past conditions of the system poorly characterize the dynamic and chaotic behavior of nonlinear dynamic systems.

The phrase is drawn from the title of the paper by EN Lorenz:

https://static.gymportalen.dk/sites/lru.dk/files/lru/132_kap...

Somber note: Weather is in fact, becoming predictable, precise and accurate: towards chaos, volatility and instability. Measurement and policy do not adequately address the ongoing issue of climate "change", which may more adequately be defined as climate chaos.

In the UK the forecasts are often very accurate up to 3 days in advance, with the hourly one accurate on 24 hours.

After 3 days there's just too much chaos normally for the models, but it does depend on the conditions, and what you are predicting.

From https://www.ecmwf.int/en/about/media-centre/science-blog/201..., more plain language discussion about low cloud at https://blog.metoffice.gov.uk/2017/04/07/why-is-forecasting-...

"Convection is complex and hard to model because it involves multiple processes, many of which take place at scales smaller than the model’s grid cell, so they need to be parametrized. The assumptions that have to be made when parametrizing convection explain many of the systematic errors in convective precipitation."

(in other words modelling physical interactions on a relatively small scale (1km) is hard)

I think the quickest answer is that weather is chaotic and incredibly complex.

https://en.m.wikipedia.org/wiki/Edward_Norton_Lorenz - under scientific career and then Numerical weather prediction as well as Chaos theory, there's a good summary

I had a meteorologist explain this to me once: weather forecasting is highly complex, near to an art form. It's not yet possible to model the forecast with precise enough rules that it can be done by machine within a reasonable compute budget. In addition, as others have noted, variations in topography can create many microclimates, so that the forecast for a given region is actually the average forecast for all the area within that region.

Very accurate forecasts for a specific spot are possible, but today require a trained meteorologist. In practice, the only time people are willing to pay for this expertise are when they're running an airport, planning a war, or launching / landing a spacecraft.

Perhaps in the future we'll have enough compute power on demand for anyone to accurately model on-demand forecasts for specific spots.

I keep a close eye on snow forecasts for the Cascade mountains in Washington state as a skier and local to me near Seattle in the winter. I don't rely solely on consumer weather apps for detailed forecasts a few days out. But I have a few sources I find are very accurate.

Weather.gov is the best forecast in my experience.

And a local resource, who relies on forecasting data, is cliffmass.blogspot.com. He is a UW professor who breaks down the available models and offers a forecast of his own when there is the potential for lowland snowfall.

Both of these sources nailed the significant snow event we just had in western Washington.

Perhaps there is still just a disconnect between the simple, automated consumer forecasts and the actual prediction models that are available.

To my understanding, it is linked to the double pendulum problem. Weather forecasting is hard because it is chaotic. https://www.youtube.com/watch?v=HPAYf2zpmOo

Elephant in the room: models are desyncing because we broke the planet.

One thing I noticed recently which breaks forecasts is the size of the area. For example city forecast on wunderground may be valid for place where you have two or more stations in a suburb that's 1km across. (With lots of data points around, going on for kilometres) On the other hand, at home, my town area is closer to 10km across, has 2 stations in it and has no surrounding context.

Which means that I may have a forecast of 24h of rain which is accurate, but my house will still be dry.

I recommend the book Chaos by James Gleick, it explains some of the limitations of predicting systems as complex as the atmosphere.

Forecasts in my area in Vermont used to be quite accurate, even a week out. Lately the range of accuracy seems to be about 1-3 days. There are several possible causes. NOAA is getting much less sensor data from commercial aircraft because planes are not flying as often. Weather has also become more unpredictable, the current polar vortex intrusion into the USA is a great example; I've been reading the forecast discussions for my area and the forecasters are less certain about my area lately because it is now on the edge of systems being pushed around by the vortex. A slight change in the wind now can make a difference between 0.4" of snow and 5" of snow here. I could imagine this affecting models also, which tend to rely on past behavior and now have to contend with a not unprecedented but still fairly radically different situation.

I've also noticed over the years that different sources of weather info are more or less pessimistic. Weather radios always had very accurate forecasts that were a little pessimistic and included some discussion of the possible variances and were used by many planners, police, fire depts, transportation companies, pilots, etc. I haven't listened much to weather radio in recent years so I don't know if that has changed. However the forecasts on the NOAA web site have become progressively more cheerful over the last 10 years or so, predicting sunny skies more often, and only changing to reflect a coming storm as it gets closer. They did change the way the "percent chance" numbers work some years ago (7+ years ago?) so they tend to be more accurate (40% chance of snow used to be an almost certainty here, now 50% is close to 50%). Forecasts, at least the ones indicated by icons and a line or two of text, have become much more optimistic only shifting to become more accurate 1-3 days out from the actual approaching events. If you read the forecast discussion you'll get a more accurate picture of what to expect, but it is pretty technical and not easily understood by your average reader without spending some time getting to know what all the acronyms and special phrases mean (also there is a tendency to use names of geological features as boundaries for weather phenomena, and not many people know these names for the whole surrounding area any more, e.g., river valleys, mountain ranges, local names for areas around larger cities, etc.)

I get the feeling (no hard evidence) that the happier forecasts might have been done for political reasons; a cheery forecast makes for a happier nation. It leaves people less prepared for severe weather, but they do still have several days warning and improves the nations mental health (not an insignificant concern).

So my guess is that lately the pandemic has reduced the amount of data they have to work with (besides the aircraft data maybe companies that ran weather stations have gone out of business or states have cut back on maintaining weather stations to save money?) and that is to blame for any recent destabilizations of predictions. Possibly the polar vortex is also responsible for uncertainty in forecasts in recent months. The cheerier forecasts could be just a calculation on the part of the service itself; it's tough to be the bringer of bad news and people naturally start to hate you for it (try doing QA on software and watch people cringe as you walk towards them with news of the latest bug in their code) so maybe they tried a more positive outlook to try to get people to treat them less like ogre's. Could be something presidents ordered also, to make people happier so they'll get re-elected. Or maybe there are technical reasons like now they rely to greater degree on models whereas before they relied more on people. Possibly whoever wrote the models built in their own biases and decided all on their own that the weather should be cheerier! Maybe forecasts now use machine learning and the training data's accuracy has faded with time or had biases to begin with. Some web research would probably turn up some history of weather models in the USA. There are probably a lot of scientific papers about the models as well.

Note also that the NOAA web site typically provides some maps which show model output in more detail and which you can scroll forward in time (I wish it was easier to see past predictions also) to see how regions are expected to change over time in air pressure, temperature, precipitation, and more. There are regional differences in forecasting too, with forecasters tending to be located near major cities where forecasts are tuned for the local area. The people behind those forecasts do change over time, and likely some aspects of the forecasts also change with them.

One thing you can do to improve forecasts in your area is to set up a weather station and send the data to NOAA. That also lets you make your own assessments and get to know the seasonal changes in the behavior of your local weather. There's been some talk of microclimate forecasts; I think that would require a big increase in the number of sensors scattered around the USA, e.g., it might be possible if all the outdoor Internet of Things sensors reported weather related data to NOAA (and they had some sense of what the accuracy of different kinds of sensors was). Maybe cars could become part of the weather sensing system.