When algorithms attack, does Google hear you scream?

Inside Google's search penalties gulag


Analysis One day in June 2006, search startup Foundem vanished from Google. Foundem is the developer of a "universal vertical search" technology, and currently offers comparison shopping across a range of consumer and travel categories by drilling down into vendor sites and returning details of actual flights and products. But on 26 June 2006, Google flipped a switch and all of the obvious comparison shopping keywords no longer applied for the company.

Eric Schmidt

Schmidt: Nothing evil to see here...

Foundem had been effectively 'disappeared' from Google search results. A few weeks later in August, it was hit with an AdWords penalty, sending the quality score which determined Foundem's minimum AdWords bid through the floor. From paying around 5p per click for a Google sponsored link, Foundem claims it went to £5. The "quality score" is an automated assessment of the AdWords advertiser's landing page, and Foundem's had been lowered from a rating of "excellent" or "very good" to "poor" or "very poor".

The company began a series of reconsideration appeals, but it took a year of trying before senior Google AdWord Evangelist Fred Vallaeys confirmed in a conference call that Foundem had been hit by new algorithms designed to penalise certain types of vertical search sites, including price comparison and travel search. Google documentation specifically refers to penalties for these classes of site, and from the time they hit Foundem until quite recently, the documentation did not appear to require price comparison sites and travel aggregators to have done anything specifically 'wrong' to incur penalties.

Although it seems clear that specific changes made in June 2006 caused substantial damage to Foundem and other sites, a Google spokesman contacted by The Register declined to discuss specific cases and specific algorithm changes, saying only that "we make hundreds of changes annually to the search algorithm".

On vertical search penalties, the spokesman referred us to a statement provided to the Guardian earlier this year, which said: "We do not penalise specific types of site. Vertical search sites are important to us and our users - indeed vertical search sites which offer added value often come top of our search rankings. We make all of our search quality assessments based solely on relevance to our users."

When The Register referred the spokesman to Google guidelines which do indicate that Google penalises vertical search sites, he told us: "It's not vertical search sites full stop, but vertical search sites 'whose primary purpose is to send users to other shopping/travel comparison sites, rather than to provide useful content or additional search functionality'."

The hand of god?

There is a widespread view that Google's rankings are untouched by human hand, crafted impartially by machine, and machine alone. But there's much more to it than that - Google's verdicts on landing page quality are Google's opinion, and the company even says as much, sometimes. The algorithms play a part in the verdict, certainly, but algorithms are produced by humans working to policies, so you could view them as a prism through which Google's opinions are projected.

Google's systems are intended to catch scammers, get rich quick sites, scrapers and so on, but that presents a problem for legitimate businesses caught by them - is your business idea so dumb that you are indistinguishable from a scammer? Is the value you thought you were adding worthless in the eyes of the world? It's all very simple for the real scammer, who can just fold his tent and start a near-identical scam somewhere else on the intertubes, but it's a lot harder for the people who're collateral damage. There's nothing that SEO (Search Engine Optimisation) can do to overcome Google's penalty, and it's hard to get any kind of answers out of the company.

Foundem search results

Foundem is acutely aware of the taint that attaches to you if you're blackballed by the world's least evil company, and brandishes awards that show that Google's opinions aren't shared by other people who might be thought to know what they're talking about. The October issue of Which? Holiday, for example, ranked Foundem third best flight comparison site in the UK, above Kayak and Kelkoo, while The Gadget Show rated it the best of the UK's 12 leading price comparison sites in December 2008.

The company has patents on its price comparison search technology, and provides services for publishing companies including Bauer Media, IPC Media and Future Publishing. So quality is very much a matter of opinion - but Google's opinion, right or wrong, is massively valuable.

Foundem's AdWords penalties were eventually removed via whitelisting in September 2007, shortly after the conference call with Vallaeys, but not before the AdWords Quality Team had twice insisted that the penalty was correct. Immediately after it was imposed in August 2006, the Team wrote: "Since the primary purpose of the site is to drive traffic to other websites, the Quality Team has decided that the initial evaluation was not in error."

And embarrassingly, on 14 August 2007, just six weeks before Google tacitly conceded by whitelisting Foundem that the evaluation had been in error: "I've confirmed that the original quality review of your site was correct, and that your current landing page quality is very poor."

Similar topics

Broader topics


Other stories you might like

  • Google has more reasons why it doesn't like antitrust law that affects Google
    It'll ruin Gmail, claims web ads giant

    Google has a fresh list of reasons why it opposes tech antitrust legislation making its way through Congress but, like others who've expressed discontent, the ad giant's complaints leave out mention of portions of the proposed law that address said gripes.

    The law bill in question is S.2992, the Senate version of the American Innovation and Choice Online Act (AICOA), which is closer than ever to getting votes in the House and Senate, which could see it advanced to President Biden's desk.

    AICOA prohibits tech companies above a certain size from favoring their own products and services over their competitors. It applies to businesses considered "critical trading partners," meaning the company controls access to a platform through which business users reach their customers. Google, Apple, Amazon, and Meta in one way or another seemingly fall under the scope of this US legislation. 

    Continue reading
  • Hangouts hangs up: Google chat app shuts this year
    How many messaging services does this web giant need? It's gotta be over 9,000

    Google is winding down its messaging app Hangouts before it officially shuts in November, the web giant announced on Monday.

    Users of the mobile app will see a pop-up asking them to move their conversations onto Google Chat, which is yet another one of its online services. It can be accessed via Gmail as well as its own standalone application. Next month, conversations in the web version of Hangouts will be ported over to Chat in Gmail. 

    Continue reading
  • I was fired for blowing the whistle on cult's status in Google unit, says contractor
    The internet giant, a doomsday religious sect, and a lawsuit in Silicon Valley

    A former Google video producer has sued the internet giant alleging he was unfairly fired for blowing the whistle on a religious sect that had all but taken over his business unit. 

    The lawsuit demands a jury trial and financial restitution for "religious discrimination, wrongful termination, retaliation and related causes of action." It alleges Peter Lubbers, director of the Google Developer Studio (GDS) film group in which 34-year-old plaintiff Kevin Lloyd worked, is not only a member of The Fellowship of Friends, the exec was influential in growing the studio into a team that, in essence, funneled money back to the fellowship.

    In his complaint [PDF], filed in a California Superior Court in Silicon Valley, Lloyd lays down a case that he was fired for expressing concerns over the fellowship's influence at Google, specifically in the GDS. When these concerns were reported to a manager, Lloyd was told to drop the issue or risk losing his job, it is claimed. 

    Continue reading
  • End of the road for biz living off free G Suite legacy edition
    Firms accustomed to freebies miffed that web giant's largess doesn't last

    After offering free G Suite apps for more than a decade, Google next week plans to discontinue its legacy service – which hasn't been offered to new customers since 2012 – and force business users to transition to a paid subscription for the service's successor, Google Workspace.

    "For businesses, the G Suite legacy free edition will no longer be available after June 27, 2022," Google explains in its support document. "Your account will be automatically transitioned to a paid Google Workspace subscription where we continue to deliver new capabilities to help businesses transform the way they work."

    Small business owners who have relied on the G Suite legacy free edition aren't thrilled that they will have to pay for Workspace or migrate to a rival like Microsoft, which happens to be actively encouraging defectors. As noted by The New York Times on Monday, the approaching deadline has elicited complaints from small firms that bet on Google's cloud productivity apps in the 2006-2012 period and have enjoyed the lack of billing since then.

    Continue reading
  • It's a crime to use Google Analytics, watchdog tells Italian website
    Because data flows into the United States, not because of that user interface

    Updated Another kicking has been leveled at American tech giants by EU regulators as Italy's data protection authority ruled against transfers of data to the US using Google Analytics.

    The ruling by the Garante was made yesterday as regulators took a close look at a website operator who was using Google Analytics. The regulators found that the site collected all manner of information.

    So far, so normal. Google Analytics is commonly used by websites to analyze traffic. Others exist, but Google's is very much the big beast. It also performs its analysis in the USA, which is what EU regulators have taken exception to. The place is, after all, "a country without an adequate level of data protection," according to the regulator.

    Continue reading
  • Google recasts Anthos with hitch to AWS Outposts
    If at first you don't succeed, change names and try again

    Google Cloud's Anthos on-prem platform is getting a new home under the search giant’s recently announced Google Distributed Cloud (GDC) portfolio, where it will live on as a software-based competitor to AWS Outposts and Microsoft Azure Stack.

    Introduced last fall, GDC enables customers to deploy managed servers and software in private datacenters and at communication service provider or on the edge.

    Its latest update sees Google reposition Anthos on-prem, introduced back in 2020, as the bring-your-own-server edition of GDC. Using the service, customers can extend Google Cloud-style management and services to applications running on-prem.

    Continue reading
  • Makers of ad blockers and browser privacy extensions fear the end is near
    Overhaul of Chrome add-ons set for January, Google says it's for all our own good

    Special report Seven months from now, assuming all goes as planned, Google Chrome will drop support for its legacy extension platform, known as Manifest v2 (Mv2). This is significant if you use a browser extension to, for instance, filter out certain kinds of content and safeguard your privacy.

    Google's Chrome Web Store is supposed to stop accepting Mv2 extension submissions sometime this month. As of January 2023, Chrome will stop running extensions created using Mv2, with limited exceptions for enterprise versions of Chrome operating under corporate policy. And by June 2023, even enterprise versions of Chrome will prevent Mv2 extensions from running.

    The anticipated result will be fewer extensions and less innovation, according to several extension developers.

    Continue reading
  • UK competition watchdog seeks to make mobile browsers, cloud gaming and payments more competitive
    Investigation could help end WebKit monoculture on iOS devices

    The United Kingdom's Competition and Markets Authority (CMA) on Friday said it intends to launch an investigation of Apple's and Google's market power with respect to mobile browsers and cloud gaming, and to take enforcement action against Google for its app store payment practices.

    "When it comes to how people use mobile phones, Apple and Google hold all the cards," said Andrea Coscelli, Chief Executive of the CMA, in a statement. "As good as many of their services and products are, their strong grip on mobile ecosystems allows them to shut out competitors, holding back the British tech sector and limiting choice."

    The decision to open a formal investigation follows the CMA's year-long study of the mobile ecosystem. The competition watchdog's findings have been published in a report that concludes Apple and Google have a duopoly that limits competition.

    Continue reading
  • FTC urged to probe Apple, Google for enabling ‘intense system of surveillance’
    Ad tracking poses a privacy and security risk in post-Roe America, lawmakers warn

    Democrat lawmakers want the FTC to investigate Apple and Google's online ad trackers, which they say amount to unfair and deceptive business practices and pose a privacy and security risk to people using the tech giants' mobile devices.

    US Senators Ron Wyden (D-OR), Elizabeth Warren (D-MA), and Cory Booker (D-NJ) and House Representative Sara Jacobs (D-CA) requested on Friday that the watchdog launch a probe into Apple and Google, hours before the US Supreme Court overturned Roe v. Wade, clearing the way for individual states to ban access to abortions. 

    In the days leading up to the court's action, some of these same lawmakers had also introduced data privacy bills, including a proposal that would make it illegal for data brokers to sell sensitive location and health information of individuals' medical treatment.

    Continue reading
  • Google: How we tackled this iPhone, Android spyware
    Watching people's every move and collecting their info – not on our watch, says web ads giant

    Spyware developed by Italian firm RCS Labs was used to target cellphones in Italy and Kazakhstan — in some cases with an assist from the victims' cellular network providers, according to Google's Threat Analysis Group (TAG).

    RCS Labs customers include law-enforcement agencies worldwide, according to the vendor's website. It's one of more than 30 outfits Google researchers are tracking that sell exploits or surveillance capabilities to government-backed groups. And we're told this particular spyware runs on both iOS and Android phones.

    We understand this particular campaign of espionage involving RCS's spyware was documented last week by Lookout, which dubbed the toolkit "Hermit." We're told it is potentially capable of spying on the victims' chat apps, camera and microphone, contacts book and calendars, browser, and clipboard, and beam that info back to base. It's said that Italian authorities have used this tool in tackling corruption cases, and the Kazakh government has had its hands on it, too.

    Continue reading

Biting the hand that feeds IT © 1998–2022