You don’t need a huge dataset to gain insight into your business. In fact, a big dataset may be detrimental according to Alastair Pearson, Lead Partner for insight analytics at PricewaterhouseCoopers.
“Most people wouldn’t know what to do with realtime big data,” said Pearson, speaking at Business Insider’s Performance Insights panel on using data in business.
“Big data is good and allows you to do great things, but if you don’t have it there are still insights to be had.”
The key is to use your dataset effectively no matter the size, to tease out the information you need from all the noise. Pearson drew this diagram to illustrate the problem that can come with a huge dataset – discovery.
On top of the statistical methods, Pearson emphasises that real insights come from combining your own datasets with public ones – hence you don’t need your own gigantic dataset.
When he was analysing commuter movements at a train station, for example, the breakthroughs came when the railway’s data was used in conjunction with publicly available datasets like the local weather. No matter how comprehensive, the railway’s data by itself wouldn’t have been nearly as fruitful.