Karl Auerbach's prediction that the internet is balkanizing into groups of people who only accept traffic from each other took another step closer to reality today. The veteran TCP/IP engineer and ICANN board member has warned of the effect for years.
"The 'Net is balkanizing. There are communities of trust forming in which traffic is accepted only from known friends," Auerbach told Wired last year.
The trend can be seen at various levels. At the user level, where we see bloggers repeating each other in an echo chamber and reinforcing their views; in the middle of the network, where Verizon recently blocking off inbound email from Europe, and it's happening deep down at the packet level too, as a result of the net's background radiation.
But all these may look like an innocent prelude. Google said today that its search engine will respect a new link attribute, "rel=nofollow", which will means its algorithms will not give weighting to the target URL. MSN, Yahoo! and blog vendors said they'll follow suit. It's effectively declaring PageRank™ dead for weblogs, in an attempt to stem the problem.
The problem is the explosion of comment spam, whereby spammers use the open comment sections of weblogs to promote their wares in the major search engines rankings. PageRank™ era, too. Google owed its success in part to the early effectiveness of link maps, but it has since demoted the factor after widespread criticism, and some embarrassing incidents. [More on that at The New York Times and Le Monde]. It's also a major blow to the 'Religion of the Hyperlink', faith in which you can see expressed in phrases like "the uniquely democratic nature of the web", coined by Google. Obviously this doesn't refer to spammers who are voting early and often, and in ever greater numbers.
Comment spam has increased exponentially since last November. (We'll explain why later this week). Like email spam, it's a classic tragedy of the commons. But other options are available, which have more predictable conseqeunces. One such is verifying the user via a "Captcha", a challenge-response system which presents the user with a graphic of a distorted word or sequence of letters, which a human can interpret but a bot generally cannot.
Not everyone believes the benefits of the nofollow link attribute outweigh the potential side effects.
"This will do very little to cull comment spam," notes ThreadWatch's Nick W. "Spammers will just redouble their efforts to hit blogs without the plugin... It could skew the web."
"Am I the only one to think that a search engine actively trying to encourage people to hide their content from it, isn't going to flaw their main aims?" observes one member of the Search Engine Watch Forum.
"If such a tag were used widespread against comments and trackbacks, then wouldn't this end up kneecaping blogs, by killing their intricate networks of interlinks?" he adds.
Other forum members renew the call for blogs themselves to be removed from the main index and placed in a separate part of Google, like Usenet forms Google Groups. The idea was first floated by a reader more than two years ago, and is very a popular solution amongst regular Google users. That would ensure the main content of Google consisted of material edited by humans, rather than the wasteland of abandoned sites wide open to spammers, that spammers naturally abuse.
"The Spammers have Won," reckons blogger Andy Wismar. "The best and brightest at Google, MSN, Yahoo!, and blog tool creators have gathered to say 'In your face, we'll just devalue EVERY link, crippling your model and our own in the process!".
Displaying not a little control-freakery, some thin-skinned bloggers - who notoriously shun dissonant views - were quick to welcome the move.
"Now I can link to things I don't like without sending Google (er, or MSN or Yahoo) love over to them," writes an excited Robert Scobie on his Microsoft blog.
"It's one of those moments that's purely good," said his former employer, Dave Winer yesterday. Winer recently suggested that 'podcasts' should be submitted to him advance so he could determine whether they contained jokes that undermined their 'authenticity'.
A sort of blog version of the FTC. ®
Harvard Man in lesbian mix-up wants satire clearly labeled
Malware, spam prompts mass net turn off
Emergency fixes for blog-clogged Google
Blog noise achieves Google KO
Google bug blocks thousands of sites
Blog noise is life or death for Google