For you sports junkies out there, if I could prove to you that I can correctly predict winners and losers with 80% accuracy, would you use my advice in Las Vegas? I don’t gamble, but according to this article on sports forecasting that summarized the results of professional research, the best methods provided only 54% accuracy. Yet, even with what one might think is an accuracy not much better than a coin flip, hypothetical bets using this model would have provided 16% returns.
Now, what if I told you that professional weather forecasts can correctly predict whether rain will fall on your specific field within the next three days 80% of the time and with almost 70% accuracy out to nine days? Considering how many risks we take on things that are less predictable than the weather, whether it be the stock market, horses, or sports, I posit that there is untapped potential in the proper use of weather forecasts to increase efficiency or productivity, and ultimately improve margins. In the case of useful rainfall predictions, there is potential to reduce risk of over-watering or mistiming a rain-sensitive product application or a host of other operations that can affect yield or operational efficiency.
Like my last article where I used independent forecast accuracy assessment from ForecastWatch to demonstrate that location-specific temperature forecasts are skillful out to around 10 days into the future, this month we use their service to demonstrate the accuracy of rainfall predictions. The data discussed here is based on an average of many different weather service providers’ daily nine-day forecasts collected each day for about 750 U.S. locations during January through November of 2017. These are not “area” forecasts where a successful forecast only means rain fell in a larger area. Rather, these are point-to-point comparisons.
For the forecasts to be considered skillful, they have to be better than forecasts created using the “persistence” technique. A persistence forecast is one that assumes the conditions occurring at the time of the forecast will continue through the forecast period. So, if it is raining at the time of the forecast, a nine-day forecast would assume that each of the subsequent nine days would have rain. Such a forecast requires no special knowledge or skill to create.
The graph below shows the average percentage of correct precipitation forecasts for all of the weather services as a function of lead time in days compared to the persistence forecasts. Indeed, the forecast services are around 80% accurate for the first three days and tail off to just under 70% by day nine, and for all days, the forecasts are better than persistence. You might be surprised, though, that the persistence forecasts are 60-70% accurate as well. This is because precipitation events are relatively infrequent when you consider the entire U.S. over nearly a full year, occurring only about 30% of the total time. These overall accuracy numbers include all of the cases where no rain was forecast nor occurred. If you simply forecast dry conditions every day for all the locations, you would still be correct almost 70% of the time.
Thus, the value of precipitation forecasts are in those situations where rain is possible. We can use different metrics that focus more on the more difficult, non-dry events that cannot rely on random guessing and still do well. In the following graph, the solid lines show such a metric called the Equitable Threat Score (ETS) for the average of the providers (solid blue) versus persistence (solid red). Perfect forecasts have a value of 1.0, and 0.0 means no skill. Professional service providers on average provide much more skillful forecasts than persistence. In other words, you would do much better betting on one of these forecasts rather than basing future actions only on how much rain has fallen.