Newswise — Forecasts about inflation, growth, and unemployment by senior economists at big banks drive critical decisions by businesses, investors, and the Federal Reserve. The stakes are high, and sometimes those forecasts turn out to be overly rosy—or too pessimistic.
Perhaps a bigger pitfall, according to new Berkeley Haas research that examined the longest-running survey of professional forecasters, is that these predictions tend to be overly precise. Forecasters reported 53% confidence in the accuracy of their forecasts, but were correct only 23% of the time, the researchers found.
“This is an elite set of forecasters—well-informed, with lots of practice, and clear standards for performance,” says Professor Don Moore of UC Berkeley’s Haas School of Business, who studies overconfidence and decision making. “We were interested in understanding the degree to which they were vulnerable—like the rest of us—to being too sure of themselves. The answer, unfortunately, is quite a bit.”
Co-written with former doctoral student Sandy Campbell, PhD 24—now a post-doc at UCLA—the study explores lesser-considered aspects of forecasting: over-certainty and over-precision.
To examine the phenomenon, Moore and Campbell examined the quarterly Survey of Professional Forecasters, conducted by the Federal Reserve Bank of Philadelphia since 1968. Given to senior economists at big banks and corporations, it asks them to predict key economic indicators for the upcoming quarter. The data are used by organizations such as Federal Reserve in economic policies like decisions on interest rates. “It really matters whether they get this stuff right,” Moore says.
The survey asks the economists to rate the probability that an indicator will land in a certain “bucket”—for example, whether GDP will grow or fall by 1% to 2%, 2% to 3%, etc.—with the total probability adding up to 100%. Moore and Campbell analyzed 16,559 forecasts; comparing them to actual indicators, they found the forecasters got it right less than a quarter of the time. Yet on average, they were 53% sure that they picked the right answer.
The problem wasn’t over-optimism: In fact, there was no consistent direction in which the economists were wrong—some thought the economic indicators would be higher, while others thought they would be lower. The problem was that they were over-precise.
The researchers did find that the more experience a forecaster had, the more likely they were to get the right answer. But at the same time, experienced forecasters were also more over-precise in their certainty, which canceled out the increased accuracy of their forecasts.
“This tells me that all of us are fallible,” Moore says. “The Fed should really be considering monetary policies that allow for some wiggle room and hedging, given the uncertainty about what the future holds.”
The good news, Moore says, is that even if any particular prediction is over-precise on its own, forecasts tend to be more accurate in the aggregate. Since there isn’t a bias towards an overly rosy or dark future, the average of the predictions tends to fall in the correct range. Disagreement between forecasters improves the confidence calibration of the aggregate, Moore says.
The findings offer broader lessons for more accurate decision making. Whether we realize it or not, every decision we make requires a forecast or thinking about probable outcomes, says Moore, who is the Lorraine Tyson Mitchell Chair of Leadership and Communication.
“It’s common to talk about overconfidence as being overly optimistic—you think the future will be brighter than it is, or you think you are better than you are,” Moore explains. Yet being overly certain you have the right answer is just as big of a problem: fixating too closely on a specific outcome can create problems in how you allocate resources or take advantage of opportunities. “You can be both overly pessimistic and too certain of yourself.”
Most of us don’t have the benefit of consulting a wide survey of experts in making decisions in business or in our personal lives, however, so we might do well to call upon some humility when it comes to making predictions, Moore says:
- Ask yourself where you might be wrong.
- Think about the relative costs and benefits of erring on one side or another.
- Be open to criticism.
“Reflecting on your vulnerability to error will increase the probability that you are well-calibrated in the confidence of your judgment, and improve your chances of making a good decision,” he says.
RSS