Foundem – the UK-based vertical search engine that sparked antitrust investigations into Google on both sides of the Atlantic – has accused the web giant of demoting vertical search competitors with the latest major update to its famous search algorithms, an update officially aimed at reducing "webspam".
In a document (PDF) posted to the web on Wednesday, Foundem does not say whether its own search engine was demoted by Google's "Panda" update – a change that went live in January – but it does say that Panda should play a major role in the ongoing investigations into Google's search and ad practices.
"In essence, Panda marks a significant escalation in Google’s undeclared war on its vertical-search rivals," Foundem says. "So far, few have made a connection between Panda and the various antitrust Investigations into Google. But Panda isn't just relevant to these investigations; it is central to them."
Foundem was not immediately available for comment. Google has not yet responded to a request for comment.
As Foundem explains in its document, Google said that Panda was a means of reducing "webspam", described as "junk you see in search results when websites try to cheat their way into higher positions in search results or otherwise violate search engine quality guidelines". This included a crackdown on "content farms" – or sites with "shallow or low quality content" – but more broadly, Google said the change would affect "sites that copy others’ content and sites with low levels of original content".
By definition, vertical search engines – like any search engine, including Google's – have "low levels of original content". Many people view vertical search engines - including price comparison sites and travel search site - as spam. But if done well, these can be valuable web services. And it should be noted that Google offers its own price comparison service and that it has big plans for travel search.
According to Foundem, Google's Panda update has demoted multiple vertical search engines, both large and small. "With Panda, Google is now targeting many established vertical search brands, as well as emerging ones," it says. "Still mindful that it cannot openly penalise well-known competitors, Panda's algorithmic demotions are more subtle than their predecessors: although affected sites do not completely disappear from Google's search results, they are systematically demoted to a point beyond the reach of most users, and so receive little or no traffic from this vital channel."
Foundem began as a price comparison engine, but it now offers travel search, jobs search, and property search as well. In 2006, Foundem's site essentially vanished from Google after the web giant introduced an algorithm that specifically demoted vertical search engines, and despite repeated appeals from Foundem, the site didn't return to Google until Foundemm took the issue public – three years later – and eventually won an audience with Google's search quality team.
Foundem was only able to survive because it began offering its service as a way for businesses to offer price comparison search on their own websites.
Though Google eventually returned Foundem to its search engine, the UK outfit went ahead with its antitrust complaint to the EU. Google offers its own price comparison service – Google Product Search, a direct competitor to Foundem – and with its complaint, Foundem accused the web giant of using its dominance in overall web search to unfairly influence vertical markets. Google's search engine controls 85 per cent of the market, according to some third-party estimates, but that number may be higher.
While Foundem was effectively unreachable through Google, Google Product Search received prime placement on the Google search engine through the company's "Universal Search" setup, and several other vertical search engines remained available through Google as well, apparently whitelisted out of the algorithm change.
"You have an overwhelmingly dominant search engine," Foundem CEO Shivaun Raff told The Register in February 2010, after it was revealed that the EU had launched an informal investigation. "If you add to that that search engine's ability to apply discriminatory penalties - they're discriminatory because some services are manually rendered immune through whitelists - and you add the ability of that search engine to preferentially insert its own services at or near the top of the search results, all of that adds up to an unparalleled and unassailable competitive advantage."
Google says that it's only aim to give the searcher what they're looking for and that, at least in the US, its algorithmic decisions are protected by the First Amendment to the Constitution. "With respect to showing our 'own' services and 'preferential treatment,' our only goal is to provide the best answer for users — and sometimes the most useful answer isn’t 'ten blue links,' but a map for an address query, or a series of images for a query like 'pictures of Egyptian pyramids.'" the company has said. "We often provide these results in the form of 'quick answers' at the top of the page, because our users want a quick answer."
After reviewing Foundem's complaint and complaints from other vertical search outfits, including the Microsoft-owned Ciao, the EU launched a formal antitrust investigation into Google last December, and in June of this year, the Federal Trade Commission began an investigation in the states.
If Google's Panda update has demoted vertical search companies that have complained to regulators, it would certainly play a major role in these ongoing investigations. Foundem doesn't say which sites were affected by Panda, but it's not shy about pointing regulators – and, well, everyone else – towards the January algorithm change.
"Where Foundem’s EU Complaint reveals Google’s smoking gun, we suggest that a detailed analysis of Panda and the events leading up to it will uncover the blueprints of Google's anti-competitive equivalent of a WMD programme," Foundem says.
"The businesses being harmed by the anti-competitive practices described in Foundem’s Complaint are not Google’s rival horizontal search engines such as Bing or Yahoo. They are the thousands of businesses that compete with Google’s other services—in price comparison, online video, digital mapping, news aggregation, local search, travel search, job search, property search, financial search, and so on." ®