Apple’s voice activated assistant, Siri, might represent the future of search on iOS devices, but for today, it is a terrible substitute for Google.Apple analyst Gene Munster tested 800 searches on Google and Siri. He said Siri only had 62% accuracy in its results. He gives it a “D” grade. He pegs Google’s accuracy at 86%, a B+.
Munster says Siri is still two years behind Google, but it has potential to be disruptive in the future.
Here’s the takeaways from Munster’s note:
- We measure Siri comprehension at 83%. On a noisy Minneapolis street, Siri comprehended 83% of our queries. We believe this is the most accurate representation of Siri’s comprehension because Siri is rarely used in a perfectly quiet setting. As a point of comparison, we performed the test in a quiet room where Siri was able to comprehend 89% of queries. Siri will need to improve from a B in comprehension to at least an A if it is to be considered a viable alternative to Google (A+), which could take two or more years.
- We measure Siri accuracy at 62%. Siri accurately answer 62% on the street and 68% in a quiet room. (See page 2 for difference between comprehension and accuracy). We believe accuracy is where Siri needs the most improvement if it wants to rival Google. Currently, we measure Google at 86% accuracy in the US based on comScore result page per search data. According to comScore, Google delivered an average of 1.14 search results pages per search over the past year. We believe this is a relevant proxy suggesting that 14% of the time, users could not immediately find that for which they were looking, thus 86% of the time, Google was accurate. In order to become a viable mobile search alternative, Siri must match or surpass Google’s accuracy of B+ and move from a grade D to a B or higher.
- Google provides 60% of answers today, 48% when iOS 6 ships. We estimate that changes to Siri iOS 6 will decrease dependency on Google by 12%, from 60% to 48%. Apple has made several significant changes to Siri on iOS 6 that will decrease Siri’s dependency on Google. First and foremost, Apple’s in-house maps app will eliminate Siri’s dependency on Google for navigation. Second, Siri’s new integration of Yahoo Sports, Open Table, Rotten Tomatoes, and Fandango will provide answers for sports scores and statistics, restaurant reservations, movie show times, and ticket purchases.