A major shift occurred in polling this election cycle. National polling stalwarts like Gallup were off by more then seven points in the weeks preceding the presidential election, while Internet upstarts like SurveyMonkey were surprisingly accurate when it came to forecasting both the popular and electoral college vote.
SurveyMonkey started running a presidential poll in the last 10 weeks of the campaign. They weren’t trying to compete with Gallup and other pollsters, they were really just trying it as a proof of concept: Were the SurveyMonkey polls as good as the phone polls?
People go to SurveyMonkey for customer, educational, non-profit, and parenting surveys. Tens of millions take surveys each week. As a site, SurveyMonkey gets around 60 million completed surveys a month.
One thing that their elections team was betting on was that people would be more willing to take a political poll if they were already taking a survey than if they got a random call from a pollster. So they tacked on a series of questions about the election at the end of some of their regular surveys.
By and large, they were right. The online pollsters had response rates of around 30 per cent, compared to less than 10 per cent for some traditional pollsters.
What’s more, they had what SurveyMonkey CEO Dave Goldberg described as a “unique cost advantage, where it doesn’t really cost us anything to do this.” Gallup spends a ton of money talking to a few thousand people. SurveyMonkey polled millions, essentially for free.
The SurveyMonkey scenario is indicative of a larger trend. Years ago, Internet polling wasn’t considered credible. Landline random digit dialling was the gold standard of polling. Now, that’s all changing.Landline phone use is declining, and polling firms are now forced to buy more mobile phone information. The call return rate is at an all-time low.
According to Goldberg, that’s the elephant in the room that traditional pollsters are reluctant to address.
“There’s like 9 per cent response rate now for phones and dropping,” he said. “You get some selection bias with people who are willing to pick up a home land line. Then for the mobile — they now have to dial half the numbers as mobile — A lot of people don’t want to answer a mobile number if they don’t know who it is.”
Even more, pollsters are having trouble figuring out the ideal mix of mobile and landline phones.If you call too many land line phones, the numbers shift to the right of the political spectrum. Too many mobile lines and the numbers shift to the left.
With the Internet, though, those biases are less pronounced. With a user base as diverse as SurveyMonkey’s, the firm is able to skirt most of the demographic problems typically affiliated with online polling.
SurveyMonkey, in their experiment, did a weighting based solely on education level. It was, all things considered, pretty accurate.
Here’s how it broke down: The day before the election, the elections team announced their final estimates based on their polls. The raw model — no weightings at all — showed Obama with a 1.9 point lead on Mitt Romney. Their second model, which weighted for “lean” undecided responses and volatility, showed Obama with a 2.7 point lead. Their third and preferred model weighted for party ID, education, and undecided voters showed Obama with a 2.1 point lead.
When all was said and done, Obama won with a 3.3 point lead.
The question now is what does this mean for political polling?
In an interview with Business Insider, Goldberg put down any speculation about SurveyMonkey transitioning into full time political polling.
“We’re not planning to enter the political polling business,” he told us. “We have a unique cost advantage where it doesn’t really cost us anything to do this. We’re getting people taking other surveys and just adding this question on a survey they’re already taking.”
Business Insider Emails & Alerts
Site highlights each day to your inbox.