Exclusive Search ad systems such as Google's AdWords and Yahoo! Search Marketing are billed as auctions. But the line between auction and non-auction is more blurred than the average netizen might think.
According to Yahoo!'s chief economist, Preston McAfee, the company is "handicapping" its auctions using an algorithmic method it calls squashing. And it had been doing so since 2007.
"When someone has a really high ad click probability, they're very hard to beat, so it's not a really competitive auction," McAfee told The Reg. "So that they don't just win [every auction], we do squashing. This makes the auction more competitive.
"It's like handicapping. We handicap the people with the high click probability."
This, McAfee said, can increase Yahoo!'s revenues. "The bidders respond by bidding higher. The one who was destined to lose is now back in the race, so they bid higher trying to displace the number one, and the number one is trying to fend them off so they bid higher too.
"We can make the competition a bit more fierce using squashing, even on keywords where there's not much bidding."
Like Google AdWords, Yahoo! Search Marketing uses a "second price auction." You end up paying only if someone clicks on your ad, and the amount you pay is determined by the bid of the advertiser ranked just below you. Prices are driven by bids, but according to McAfee, squashing drives up the bidding.
Yahoo! published a research paper on the practice in 2007. But its blog post discussing the paper does not say whether the technique has actually been deployed. McAfee would not say how much squashing the company does, but this is constantly changing. "Not only are we engaged in squashing but we are currently doing a squashing refresh, resetting the parameters," he said.
On Thursday afternoon, at a press event at Yahoo!'s headquarters in Sunnyvale, California, company CTO and former chief search architect Raymie Stata told us said that the practice must be balanced against efforts to keep ads relevant and satisfy users. In essence, the degree to which Yahoo! "squashes" can be increased or decreased as the company sees fit.
However much Yahoo! squashes, it won't do so for long. Yahoo! is in the process of moving its search ad setup to Microsoft's AdCenter system, and the move should be completed later this fall. McAfee said he did not know "for sure" if Microsoft uses an algorithm similar to Yahoo!'s squashing setup. We asked Microsoft if it's using such algorithms, but it did not immediately respond.
When we specifically asked Google if it does something similar to squashing, it did not respond.
If Google does something similar to squashing, it is a potentially thorny issue for Mountain View, which controls an estimated 85 per cent — or more — of the search market. When critics accuse Google of violating antitrust law, one of its chief defenses is that because AdWords is an auction, it doesn't set prices. But if Google is squashing, this would at least call those claims into question.
Last month, Santa Clara University law professor and tech-law blogger Eric Goldman questioned whether Google was undermining the integrity of the auction process by bidding on its own auctions.
The original search-ad auction model — popularized by Overture, which was eventually bought by Yahoo! — ranked ads on the page strictly according to the amount of money an advertiser was willing to bid. But Google found a much more successful business model when it began ranking ads according to click-through rate — how likely an ad was to receive a click. In essence, it ranked ads by multiplying bids by the ad's "quality score", a calculation based on click-through rate.
"[Quality Score] looks at a variety of factors to measure how relevant your keyword is to your ad text and to a user's search query," reads Google's current AdWords help page, adding that it is "broadly determined" by three main factors: click-through rate, relevance, and landing-page quality.
With the introduction of Panama, Yahoo! mimicked the Google model. But it has introduced an added twist with squashing. "The old Overture method said rank by bid... The Google method ranked by bid [multiplied by] click probability," McAfee said. "Let's say you raise click probability to the power of gamma. What makes you the most money is a gamma between 0 and 1. The right thing from a revenue perspective is somewhere between Overture and Google. This is called squashing."
McAfee said that squashing is separate from calculations involving its own version of quality score — though the introduction of things like "relevance" and "landing page quality" can have similar (if limited) effects. "Using relevance scoring may accomplish a certain amount of squashing, but it need not," he explained.
"Squashing is aimed at making marketplaces that are not very competitive a bit more competitive, by handicapping, much the same as a handicap makes a golf tournament more competitive. Relevance scoring is aimed at incorporating the user experience into the evaluation. Yahoo does this as well using quality scoring.
"These two have quite distinct purposes, but their effects can be similar because both quality adjustments and squashing move away from a pure revenue-maximization model. The effects are similar when bidders with unusually high click rates typically have lower quality scores. Otherwise, the effects are different."
In Yahoo!'s blog post discussing Pennock's research paper, the company echoes Raymie Stata, saying that squashing must be used judiciously. "Squashing can significantly improve revenue, at the expense of advertiser and user satisfaction," the post reads. "As a result, it is necessary to set acceptable thresholds for loss of relevance, and then optimize revenue based on those thresholds." Squashing was developed by Yahoo! researcher Dave Pennock and other Yahoo! colleagues.
Asked if he believes Google is doing something similar to squashing, McAfee gave the only answer that anyone outside the Googleplex can give to such a question. "I have no way of knowing one way or another," he said. "It's very hard to know what they actually do." You can bid on an auction, but you never have access to the bids of others. ®