There will be an unprecedented focus on the funding and management of Australia’s official statistics bureau, the ABS, after the country’s first attempt to take a population census online ended up with the website crashing and the agency apologising for the fiasco.
The ABS had promised that the online form submission could handle a million entries per hour, but the website crashed in an apparent “denial of service” attack.
The ABS conceded it was unable to handle the demand in a tweet:
ABS & Census website are unavailable. The service won't be restored tonight. We will update you in AM. We apologise for the inconvenience.
— Australian Bureau of Statistics (@ABSStats) August 9, 2016
The event was so huge that the hashtag #CensusFail became the biggest trending topic worldwide on Twitter. Worldwide!
People all over Australia trying to fill out their census forms, as they are required to do by law, found themselves unable to access the census website on Tuesday night.
(Around 8pm, Australian Eastern Standard Time, Business Insider tried to access the census site and received an error message.)
It’s completely unclear where the fault is, and it may be a combination of issues, but IBM is sure to face some questions after it was awarded a $9.6 million contract to support the project.
The knock-on effects are deeper, and there are two big, uneasy questions. The first is Australia’s ability to track accurately what’s happening in the economy, a total requisite for business the modern world, through the ABS.
And then there’s the whole issue of public confidence in technology’s ability to deliver services that were being done just fine by actual people.
First to the measurement deficit.
The economics and finance community has, over the past two years, been questioning the reliability of all sorts of data from the Australian Bureau of Statistics, especially its employment market data. This has become a lost cause; the ABS jobs numbers are now largely ignored by the market as an indicator of the health of the Australian economy. In what feels like a response to this criticism, the ABS now tries to emphasise “trend” over “seasonally adjusted” data in its monthly statements on employment. It used to be the other way around.
The ABS surely realises that people dig into the details and the emphasis in a press release is of no concern to people seriously trading or thinking about investing in the economy. Surely.
Then there’s consumer price inflation data – the major driver of monetary policy decisions by the RBA and a central concern to markets since the global financial crisis – which is only released every three months. Many advanced economies like the UK and the US get inflation data, including on wages, on a monthly basis.
Why doesn’t this happen in Australia? Are we short on statisticians, or funding for the ABS? The answer here is pretty clear.
On to the service delivery question. You might remember that one of the four pillars of Malcolm Turnbull’s innovation agenda was “government as an exemplar” — the principle being that government departments and their services should set themselves the highest standards in terms of delivery to their customers.
Given the amount of money everybody hands over from their salary in taxes, when the government expects you to fill out an online form, it’s fair to want that form to work. Right?
This is the standard that every smart entrepreneur sets themselves when thinking about their customers. The ABS, already with some reputational questions hanging over it, has spectacularly failed on “government as an examplar” mission.
Separately, there are questions about the technology infrastructure. And here’s where it gets complex.
The cloud problem
The Australian census would have been supported by “the cloud”, a service which backs up all of the online activity people do every day such as banking, website hosting, Facebook, shopping.
“The cloud” is sold to people the world over as a much more cost-effective and efficient way of storing and managing their data, and hosting their websites. The contracts for cloud services — where the competition involves the likes of huge providers such as Amazon, Microsoft, IBM, Cisco and, in Australia, Telstra — run into huge amounts of money – potentially hundreds of millions of dollars.
Governments are one of the biggest customers for these services, which Gartner predicts will be a $US207 billion sector this year. When it fails this spectacularly for huge amounts of people, the fallout in terms of confidence in cloud services cannot be dismissed.
We all use cloud services every day, sometimes without realising it — for storing photos, apps, and work projects. People also hook into the cloud for their software and security updates from various providers ranging from Microsoft to Apple and, at a software level, from Salesforce to Atlassian.
Private companies are able to run these cloud services fairly seamlessly, so it will shock and annoy people that a simple form cannot be filled out and submitted to a government department.
What will infuriate many in the technology industry is that capacity for the inevitable demand should have been one of the prime considerations and testing routes in the design of the whole project. Almost embarrassingly, this matches up with what people filling out the census would expect: to go onto the website, fill out the form, and be done.
Somehow, it all fell apart. And it is not without consequences.
The data from the census, which in Australia is taken every five years, is a central pillar in business planning for Australian companies and overseas investment planners assessing the opportunities in Australia. It is also hugely important to informing the targeting of government policy priorities, as its data is folded into Treasury projections for the federal budget, and relied on by a range of government departments for their advice to ministers.
It’s used by various private and non-government organisations to help inform their policy settings and lobbying agendas, so any doubt over the credibility of the census data can have widespread implications.
There is now a period in which people who were unable to submit their data will still be able to complete the form, and the ABS says people will not be fined for failing to complete the census.
But the glitch will make statisticians, economists, and market researchers nervous about the reliability of the data. And it will put unnecessary but fierce pressure on Australian executives who are seeking to move their company’s IT platform away from home base.
*This post has been updated to the reflect the ABS attributing the outage to a “denial of service” attack.
Business Insider Emails & Alerts
Site highlights each day to your inbox.