An open source Google - without the ads

But is it legal?


With the hope of returning at least one corner of the web to its non-commercial roots, Google watcher Daniel Brandt, who curates the NameBase archive, has released the source code to a Google scraper. Brandt has been making an ad-free proxy available for two years using Google's little known minimal "ie" interface. By using this proxy, users bypass both Google's notorious "2038" cookie (that's when it expires) and the text ads.

Brandt fully expects Google to throw legal and technical resources at him, but says he welcomes the challenge if only to clarify copyright issues. Google took people's free stuff and made a $50 billion business from it, he argues.

"The commercialization of the web became possible only because tens of thousands of noncommercial sites made the web interesting in the first place," he writes. "All search engines should make a stable, bare-bones, ad-free, easy-to-scrape version of their results available for those who want to set up nonprofit repeaters. Even if it cuts into their ad profits slightly, there's no easier way to give back some of what they stole from us."

He explains in more detail in the source code: "Legally, Google probably has the right to block anyone they want. And legally, we believe that as a tiny nonprofit with an interest in Google's violations of privacy, we have the right to access Google's publicly-available data any way we want. If you want to argue about copyright, then let's start with the fact that Google scrapes billions of web pages and doesn't ask permission before making the cache copies available. Thiss craping is used as a carrier for the ads that make Google stinkin' rich.

"Now that, in our opinion, is an interesting copyright issue. As this is written, Google has a market cap of $55bn. This exceeds the market cap of General Motors and Ford combined. Google is probably the single largest information resource on the planet, and they're getting rich off of us. It's time for Google to give something back to the public sector."

The source code, which runs on Linux, asks the users only to use the program for non-commercial purposes.

"We think it would be splendid if scraping Google for nonprofit purposes, and stripping out their wretched advertising, was established someday as an acceptable, legal practice."

In the week since it launched, the source code has been downloaded about a hundred times a day says Brandt.

Google would rather you licensed its beta Web API. However, as Charles Ferguson writing in MIT Technology Review noted recently, the service is "laughably limited" to 1,000 queries a day, and offers little functionality; Google has let the offering languish.

You can find the code here [ZIP archive, 16kb], an explanation here and try out the proxy here. ®

Related stories

Google exposes web surveillance cams
Major flaw found in Google Desktop
Google News' chief robot speaks out
Gates: PC will replace TV, TV will become a giant Google
Google Desktop privacy branded 'unacceptable'


Other stories you might like

  • North Korea pulled in $400m in cryptocurrency heists last year – report

    Plus: FIFA 22 players lose their identity and Texas gets phony QR codes

    In brief Thieves operating for the North Korean government made off with almost $400m in digicash last year in a concerted attack to steal and launder as much currency as they could.

    A report from blockchain biz Chainalysis found that attackers were going after investment houses and currency exchanges in a bid to purloin funds and send them back to the Glorious Leader's coffers. They then use mixing software to make masses of micropayments to new wallets, before consolidating them all again into a new account and moving the funds.

    Bitcoin used to be a top target but Ether is now the most stolen currency, say the researchers, accounting for 58 per cent of the funds filched. Bitcoin accounted for just 20 per cent, a fall of more than 50 per cent since 2019 - although part of the reason might be that they are now so valuable people are taking more care with them.

    Continue reading
  • Tesla Full Self-Driving videos prompt California's DMV to rethink policy on accidents

    Plus: AI systems can identify different chess players by their moves and more

    In brief California’s Department of Motor Vehicles said it’s “revisiting” its opinion of whether Tesla’s so-called Full Self-Driving feature needs more oversight after a series of videos demonstrate how the technology can be dangerous.

    “Recent software updates, videos showing dangerous use of that technology, open investigations by the National Highway Traffic Safety Administration, and the opinions of other experts in this space,” have made the DMV think twice about Tesla, according to a letter sent to California’s Senator Lena Gonzalez (D-Long Beach), chair of the Senate’s transportation committee, and first reported by the LA Times.

    Tesla isn’t required to report the number of crashes to California’s DMV unlike other self-driving car companies like Waymo or Cruise because it operates at lower levels of autonomy and requires human supervision. But that may change after videos like drivers having to take over to avoid accidentally swerving into pedestrians crossing the road or failing to detect a truck in the middle of the road continue circulating.

    Continue reading
  • Alien life on Super-Earth can survive longer than us due to long-lasting protection from cosmic rays

    Laser experiments show their magnetic fields shielding their surfaces from radiation last longer

    Life on Super-Earths may have more time to develop and evolve, thanks to their long-lasting magnetic fields protecting them against harmful cosmic rays, according to new research published in Science.

    Space is a hazardous environment. Streams of charged particles traveling at very close to the speed of light, ejected from stars and distant galaxies, bombard planets. The intense radiation can strip atmospheres and cause oceans on planetary surfaces to dry up over time, leaving them arid and incapable of supporting habitable life. Cosmic rays, however, are deflected away from Earth, however, since it’s shielded by its magnetic field.

    Now, a team of researchers led by the Lawrence Livermore National Laboratory (LLNL) believe that Super-Earths - planets that are more massive than Earth but less than Neptune - may have magnetic fields too. Their defensive bubbles, in fact, are estimated to stay intact for longer than the one around Earth, meaning life on their surfaces will have more time to develop and survive.

    Continue reading

Biting the hand that feeds IT © 1998–2022