A panel on search engine spam in San Francisco this morning got extremely heated, with a Google engineer openly accusing Microsoft of copying its search results. In response, a Bing engineer said that Google should look at why search spam exists, and basically accused the company of profiting from spam sites
The panel was moderated by Vivek Wadhwa, whose TechCrunch article calling for a “better Google” kicked off a firestorm of debate about search engine spam. Matt Cutts, principal engineer at Google, who is in charge of the company’s anti-spam efforts, kicked off the discussion by saying that Bing was copying results from Google’s queries from long-tail search results.
Harry Shum, VP of search development at Bing, responded by admitting that Google had uncovered a new form of search fraud, and said he wished Google had spoken to Microsoft about it before taking it to the press. “These were a few outlier examples constructed very creatively,” he said, and not an indication that Bing is copying Google search results wholesale.
Then the conversation turned to search spam, and Shum basically accused Google of playing both sides of the spam game — on one hand, Google wants to make its search site as useful as possible so users don’t turn to alternatives. On the other hand, a lot of spam sites make their money by running ads provided by Google, from which Google takes a cut. As Shum put it:
I’d say you are really sidestepping the big problems, the origin of the spam, why they appeared in the first place. There must be an economic incentive to create his kind of content. Why? 70% of those pages show Google Ads….You can’t just say because you don’t report to VP of ad sales you have no problem.
Cutts said that Google monitors spam regardless of whether sites use Google Ads or not, and when it kicks a site out of Google’s search results it also removes it from the advertising program. He also noted that sites have other economic incentives to try and game search results — porn and casino sites, for example, pay a bounty to third-party sites who guide users to them.
Cutts also noted that Google values long-term loyalty over short-term gain, pointing to the search engine’s refusal in the early 2000s to use paid inclusion — where sites could actually pay to be included in algorithmic search results — when competitors like Microsoft and Yahoo were happy to do so.
Rich Skrenta, cofounder and CEO of alternative search engine Blekko, called the current state of search a “swamp,” and pointed to a different problem.
When Google (and Bing) test search queries, they use low-paid contract workers to check relevance. Those users might type in a query like “how to build a brick wall,” then see a result with almost exactly the same question as its title. The page actually contains no useful content, but the tester doesn’t realise that until it’s too late — they’ve already clicked through.
This panel ended about a half hour ago, but the day will be filled with other speakers talking about the present and future of search. A live video stream of the event is available here.