Big data is a hot young technology but it can really send a company down the wrong track, warns Lara Lee, COO of product-design company Continuum.Continuum is a 200-person, 30-year old Boston firm responsible for products like Procter & Gamble’s Swiffer, the One Laptop Per Child laptop, and others.
Lee is alarmed by how corporate America is using big data. Tools like Hadoop, Splunk, and ClearStory can sift through massive amounts of tweets, system logs, documents, and videos to discover things about their customers.
But it can cause companies to overlook good business opportunities. And some of these tools behave in creepy ways.
“Big data can tell you where to look,” says Lee. “It can’t tell you what to do. Big data can give us a false sense of accuracy. Where do you allow the algorithm to tell you what’s going on?”
For instance, an insurance company’s big-data system uncovered a moderate-income family who had bought very expensive insurance. Its recommendation: Ignore the anomaly. But talking to the family uncovered that the dad was a cancer survivor who feared not having adequate coverage. Because someone talked to the family, the insurance company developed a new program for other survivors and found a market that big data would have missed.
The way some big-data systems are used cross the line into spying. The classic story is when Target exposed a teen pregnancy. It saw that she bought a certain combination of items and sent her a bunch of maternity coupons in the mail, sending her enraged father to chew out a store manager. (He later sheepishly apologized: His daughter really was pregnant.)
The backlash for this kind of privacy invasion is coming. Internet Explorer 10, which ships with Windows 8, will have “do not track” features turned on by default. As we share less, big-data systems will become even less reliable, Lee warns.
The solution is pretty simple. Use big data to uncover the anomalies. Then have a human being pick up the phone or get in the car and treat this anomaly as a human being and find out what is really going on. That anomaly might be the first of a whole new set of customers.