Forecasting is a mug’s game.
Anyone who ever sat down in front of a client and made a forecast – on the economy, markets, growth, margin, yield, or any other measure – at some point in the future knows in their heart the chances of hitting a bullseye are often negligible at best.
But how does a business, a central bank, or a treasury department try devise any sort of strategic framework for informing decisions without first making forecasts?
That’s the topic of Warren Tease, principal adviser at the Australian Treasury’s macroeconomic conditions division, in an address to the Committee for the Economic Development of Australia yesterday.
Tease, who is currently leading a range of changes to forecasting processes at Treasury, said: “Treasury’s macroeconomic forecasts are a key input into the policy process. They frame budget discussions and decision making, identify emerging themes or risks to the economy that may require a Government response and are a vehicle to inform the public and the media about the likely path of the economy.”
Crucially, he added, “like all economic forecasts they have a wide margin of error.”
That sounds like a cop out. And given it was the second paragraph in his speech, had he not been speaking to an informed audience at CEDA, he might have cleared the room, particularly given that what he said next appeared to give him, and his colleagues at the Australian Treasury, a get out of jail free card.
“Practitioners are very forgiving of forecast errors as they accept the difficulty of predicting the future,” he said.
Tease took the very vexed question of forecast errors and said that “the implications of forecast errors can be more significant for public-sector forecasters as they can have material effects on the outcomes of Government decisions.”
Shareholders, business owners, management, bondholders, investors, and other users of these rubbery forecasts might suggest they are impacted as well.
Treasury looks at its forecast track record and practices around it objectively. He noted forecasters around the world have been getting it wrong since the GFC, saying that “a key feature of the forecasts for the global economy and for Australia has been the persistent downgrading of forecasts.”
Tease’s explanation for these consistent forecast errors was that things are a bit more volatile these days than they were in the past. He argued:
What is also likely in my view, but difficult to prove, is that there is a natural bias in forecasting to assume that the past will be a reliable guide to the future. This is not a bad assumption most of the time but will bias forecasts in periods like the present when we are dealing with not one but a range of unusual forces.
That does sound a bit like an excuse – Monopoly’s get out of jail free card.
But Tease explained he only recently joined Treasury with a remit from Secretary John Fraser to review its forecasting process.
Hopefully the questions Fraser asked him to answer were along these lines: If this post-GFC economic and market volatility is the new normal then how can governments, and us in Treasury, be held accountable for our forecasts? What is our strategy to deal with this increase in volatility the way money managers, or even business owners, and corporate CEO’s operating in economy must?
It seems that mightn’t be far from the truth because Tease said “my focus was not on recent forecasting performance but instead was focussed on the forecasting processes in place in Treasury and whether they were structured to produce reliable forecasts.”
Tease said even though Treasury’s forecasts in the past had been unbiased, they were as good (or bad) in terms of accuracy as other agencies, and usually missed turning points in the economy (like everyone else). But new volatility was a big challenge.
“Digging a little deeper into this quantitative work I would also observe that Treasury’s forecasts did well in normal times but were challenged when conditions were unusual,” he said.
So, to the big question.
How does Treasury plan to manage uncertainty around the economy and its forecasts?
Hint: it doesn’t.
More on that later.
Tease said best forecasting practice “is that as wide a range of inputs as possible should be used in the process. In effect casting the net as wide as possible will help manage, but not eliminate, uncertainty around the outlook.”
But Treasury practice was too narrow, he believes, because “one of the main conclusions of my Review was that Treasury’s current approach relies on fewer sources than many other public-sector forecasters. It relies predominantly on a small number of single-sector models”.
The problem is these models “are not designed to efficiently incorporate all of the linkages in the economy. So, as a tool, their ability to cope with changes in economic circumstances and shocks is constrained”.
Perhaps insiders were aware of this, but this sounds like a shocking and remarkable revelation: that the Australian Treasury, the primary body for advising the government on the economic outlook and the likely impact on finances, doesn’t have economy-wide linkages in its models.
The good news is that Tease is going about trying to fix it. Treasury has accepted his recommendations to broaden and improve process. It has also started recruiting more experienced operatives.
Here are some of the steps they are taking:
- Senior Treasury officers from a wider range of areas will be involved in forecasting;
- There will be a panel of external experts to review forecasting techniques;
- There is a new project that Tease is leading to gauge the impact of the financial sector and incorporate it into forecasting, and
- Liaison with business will be extended, including engaging with more private sector commodity analysts “to further increase the sources of information on the risks around the world economy and commodity prices”.
That last point suggests Treasury might be feeling stung by missing the scale of the decline in the terms of trade and how that has crushed the budget bottom line.
But it’s outcomes that matter.
One thing Tease said – something the Australian Bureau of Statistics won’t like – is that Treasury is mimicking the Atlanta Fed’s GDP Now reading for the US economy by building its own “nowcast” for Australian economic growth levels.
That way it gets a “timelier read on the economy”.
But uncertainty is still high and Treasury knows it’s not going to get things right all the time.
So Treasury wants to put more emphasis on the need for people not to rely too heavily on its forecasts.
Tease said: “While it is natural that the point estimate of the budget outcome attracts most of the attention it is in reality only one of a wide range of possible outcomes. This is illustrated in the following chart.”
He said that these illustrations “deserve more attention as they are a better reflection of the possible range of economic and budget outcomes.”
Tease concluded by saying forecasting is hard and forecasts are often wrong.
True, it’s a mug’s game, but he added “the process of forecasting is not just about generating point-estimate outcomes, even though they attract most of the attention, it is also about exploring the uncertainties around the outlook. As policy, business and investment decisions are all about probable outcomes in an uncertain world that is no bad thing.”
If Treasury wants Australian businesses, the population, investors, Australian dollar traders, and buyers of government debt to focus on the wide dispersion of outcomes then both they and their political masters must then stop with the point forecasts for the budget deficit and other outcomes.
To do otherwise would be misleading, and continue to put Treasury and the government in the position it has found itself where it has to protest being misunderstood.