The Forecast Folly: Why financial experts rarely stand the test of time Present success does not indicate future outcomes

The Forecast Folly: Why financial experts rarely stand the test of time

  1. Present success does not indicate future outcomes
  2. The Power of Other Opinions
  3. What do experts say about predictions?
  4. Two types of experts
  5. In Brief

Present success does not indicate future outcomes

Predictions are a form of guessing, where sometimes the guessing is more educated than others, and a more educated guesser is not necessarily more accurate. Market predictions, in particular, tend more towards the guessing end of the spectrum with the more educated and higher profile prognosticator typically carrying more weight in public opinion. Particularly at the end of every year there are a lot of predictions about what the major changes and trends are in the markets, technology, the economy, politics and life in general. This tendency to predict is further augmented at the end of a decade or century when all manner of experts believe that even though they have a poor record on telling us what will happen in the next year, they certainly can tell us what will happen in the next decade or century. Plus, no one will remember what they said 10 (or 100) years ago anyways.

The Power of Other Opinions

And the voices of others do influence the formation of our own opinions. As an example, allow me to present a trivial example, but one that occurs ALL THE TIME in my household, and unscientifically illustrates the power of other voices as we form our own ideas. In my home, I have a certain way I like to load the dishwasher. Our dishwasher has the utensil rack on the top so you lay your utensils on the side instead of sticking them in a basket. For years, I have tried to get my wife to put the spoons in one section, knives in another, forks in another. That way, unloading is easy, grab the group, stick it in the proper place in the drawer, no sorting. Nope, dumb idea she says, stop being lazy and sort them when you put them away. Then she visits a friend’s house, where they use my patent pending dishwasher loading technique. After her visit my wife comes home to show me the proper way to load the dishwasher. Spoons in one section, knives in another, forks in another. Brilliant. Even my wife needed the confirmation of an ‘expert’ whose sole expertise was being more OCD than I am. Once there was confirmation, it became a good idea.

What do experts say about predictions?

When I read or hear experts telling me what is going to happen, I am reminded of the book Expert Political Judgement by Philip E. Tetlock. It is one of the best books written on the subject of forecasting and quantifying the ability of expert prognosticators. While it was written in 2005, I have returned to it many times in my career as I remind myself to discount much of the noise out there; noise that has been amplified since the dissemination of opinions has been democratized by the internet and social media and as television shows try to set themselves apart by having polar opposites argue a point just to get controversy.

Expert Political Judgement is a culmination of Tetlock’s 20-year study on the accuracy of political forecasting. If you haven’t read it, I highly recommend it, particularly as you are being bombarded with experts telling you what will happen in 2020 and beyond.  The academically focused text isn’t geared towards the casual reader, but there are plenty of summaries available. A few key findings:

  • Experts are no better at prediction than dilettantes or the population at large (or a hypothetical chimpanzee)
  • Those who draw on many sources of information and ideas for opinions are better prognosticators than those who are more dogmatic
  • Popularity and education of the expert does not increase their accuracy

I found an interesting, if dated, article called Analysis Of Forecast Accuracy in the Political Media, published by what appears to be a graduate team at Hamilton University after 2008. In it, the authors actually hold various public figures accountable for their prognostications. The research was based on Tetlock’s work. What is fun about it is that the authors call out those public figures who make predictions and sort them into categories of those that were good at forecasting, versus those that weren’t. While it would be great if this research was more robust (and more current), there is something for both sides of the political spectrum to like here. But keep in mind, this research is over 10 years old now and those on this list may be better or worse predictors now, and many have faded into political prognostication irrelevance.

Ok, I apologize to readers on the right side of the political spectrum as this list is probably a less fun for you than I let on and surely less fun than it is for those on the left side of the spectrum. I mean, Pelosi and Schumer as ‘Good’ and Graham as ‘Bad’ is surely more enjoyable for the left than Paulson and Brooks as “Good’ and ‘Levin’ as ‘Bad’ are for the right. But I didn’t have a lot to work with as the study was old and I was limited to the personalities they examined. It was not my intention to give you less joy in this new year article. The bottom line is everyone is pretty bad and these results wouldn’t hold up to the rigor of Tetlock’s study.

Fun for LiberalsFun for Conservatives
Good PrognosticatorsPaul Krugman
Nancy Pelosi
Chuck Schumer
Hank Paulson
David Brooks
Kathleen Parker
Bad PrognosticatorsNewt Gingrich
Mike Huckabee
John Kerry
Hillary Clinton
Do the OppositeLindsay Grahm
Cal Thomas
Joe Lieberman
Carl Levin

Source:https://www.hamilton.edu/documents/Analysis-of-Forcast-Accuracy-in-the-Political-Media.pdf

Ok, I apologize to readers on the right side of the political spectrum as this list is probably a less fun for you than I let on and surely less fun than it is for those on the left side of the spectrum. I mean, Pelosi and Schumer as ‘Good’ and Graham as ‘Bad’ is surely more enjoyable for the left than Paulson and Brooks as “Good’ and ‘Levin’ as ‘Bad’ are for the right. But I didn’t have a lot to work with as the study was old and I was limited to the personalities they examined. It was not my intention to give you less joy in this new year article. The bottom line is everyone is pretty bad and these results wouldn’t hold up to the rigor of Tetlock’s study.

The same lack of expert prediction acumen can be found in the world of finance. With headlines and research such as “Why do we bother with stock market forecasts”, and “Market Forecasts: Rarely Accurate and Never Reliable”, there is a negative sentiment for predictions. The same goes for economics, where despite awards given for accuracy, articles such as “Why Economic Forecasting Has Always Been A Flawed Science” and “Why Economists’ Predictions Are Usually Wrong” convey a general skepticism for the sooth saying abilities of economists in general. But we continue to seek these experts out as sources of expertise, particularly when their opinions resonate with our own.

More specific research has been conducted on equity research analysts. An article in Forbes from March 2019 that was primarily reporting on Bill Gross’ retirement from Janus (and the his recent missed predictions) also contained this tidbit on some older research:

“Boris Groysberg, of the Harvard Business School, looked at the transition of 1,000 star equity analysts across firms (a group whose performance is profoundly transparent as they make public stock recommendations). A star was defined as a sector analyst ranked in the top four by Institutional Investor.

He found that not only does the star’s performance drop when they change firms, but so does the performance of their new associates.

In short, even when the analyst shows great ability in their specialty area, that ability is also dependent on the structure that supported them as they made their stock predictions. Moving the analyst does not move the performance of that analyst.

Two types of experts

Returning to Tetlock’s research, he divided the world of experts into two categories: Foxes and hedgehogs. Foxes, are those who are constantly adjusting their worldview to new data and hedgehogs are those that form opinions through the lens of their deeply held beliefs, often pushing away contrary ideas and having ample confidence in their worldview. Perhaps unsurprisingly, foxes are better forecasters than hedgehogs.

But hedgehogs make better pundits as forcefully arguing one’s point and refuting all evidence to the contrary can be, from the right hedgehog, very convincing or at least entertaining. And it seems we see more of this type of pundit on TV. No one wants to see someone couching their opinions or forecasts in nuance and uncertainty.

In Brief

So, as we begin a new year and decade, try to remember that all those voices telling you where you should put your client’s money, what the market is going to do, who’s going to win the next election, and everything else, is done by people with no more ability to tell the future than you. But many of them have done great research that you can learn from and can help inform your worldview.

In short, be a fox.

You can read some of our economic predictions, for what they’re worth, at our blog, rixtrema.com/blog. We’ve covered Brexit, IMF Global Growth Expectations, and are beginning to cover Regulation Best Interest. If you have any questions about our financial planning software platform, then please contact our Client Success Team at clientsuccess@rixtrema.com

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.