STUDY: Computer Investment Modelling Is Now A Pathological Problem In The Financial Industry

Bubble magician Michele Cafaggi.
Ian Gavan/Getty Images

Getting that investment portfolio mix — the balance between income and capital growth, local stocks against international, direct property vs listed vehicles — just right is critical.

And it’s probably best left to a professional to work that out according to what you want, your stage of life and what your goals are.

So it would seem to be an excellent idea when an adviser pulls out powerful software and data going back 20 years to decide which portfolio would be best.

Not necessarily so. There are pitfalls to looking at thousands of simulated portfolios and measuring their performances against return on risk to find the one which best fits an individual.

The portfolios which look best in this form of modelling are actually sometimes those which are extremes in datasets. This problem is known as “backtest overfitting”, and its perils are dissected in an article, “Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance”, in the May 2014 issue of the journal Notices of the American Mathematical Society. The authors are David H. Bailey, Jonathan M. Borwein, Marcos Lopez de Prado, and Qiji Jim Zhu.

“Recent computational advances allow investment managers to methodically search through thousands or even millions of potential options for a profitable investment strategy,” the mathematicians write.

“In many instances, that search involves a pseudo-mathematical argument which is spuriously validated through a backtest.”

The mathematicians suspect that a large proportion of backtests published in academic journals may be misleading.

“The situation is not likely to be better among practitioners,” they write. “In our experience, overfitting is pathological within the financial industry.”

Later in the article they say: “We strongly suspect that such backtest overfitting is a large part of the reason why so many algorithmic or systematic hedge funds do not live up to the elevated expectations generated by their managers.”

Of course, financial advisers usually preface everything by stating: historical performance is no indicator of future returns.

Mark Rantall, CEO of Australia’s Financial Planning Association, says there are a few rules.

“The first rule is that if it sounds too good to be true it usually is,” he says. “The second rule is to diversify your risk by diversifying your portfolio and the third golden rule is past performance is no guarantee for future performance.”

And the other thing to remember is that only a small percentage of the top performing companies 20 years ago are still in existence. They get taken over, they go out of business, trends change.

“You only have to look at traditional shop front retail businesses and how technology and online sales has overtaken some of them,” Rantall says.

“Business propositions which worked 20 years ago, don’t necessarily work today. It’s a guide but you don’t get hooked up on past performance. You look at the environment for a company and how it is going to perform. It could have been stellar last year and it could be a lame duck next year.”

The mathematicians make a good point, says Dr Andrew Grant of the University of Sydney Business School.

“It is a useful thing to know. However, people in finance have been aware of this problem for quite some time,” he says.

“You can find data which supports your argument if you conveniently ignore the data that doesn’t support your argument.”

However, most investors switch off when you start telling them about back-testing, historical data or statistics.

“You can’t use that in marketing,” Dr Grant says. “We learn from psychology or behavioural economics that investors under-react to abstract information or things they can’t really comprehend or things they don’t deem to be important.”

Generally, investors gravitate to the fund which did well last year. That may or may not be the best strategy but they look at what did well in the past and follow.

“I think the point is that funds aren’t rewarded for providing this information and they are not punished for not providing it,” he says. “In either case there is no incentive for them to do so.”

And any strategy which works tends to run down over time, become less effective, as others learn about it and follow the “new” path.

“There’s no guarantee that what worked in the past will work in the future,” Dr Grant says. “articularly with algorithmic trading there’s a lot of copycats going on.”

Business Insider Emails & Alerts

Site highlights each day to your inbox.

Follow Business Insider Australia on Facebook, Twitter, LinkedIn, and Instagram.