Google launches Octane JavaScript benchmark suite

Shocker: Chrome leads the the pack in new tests


Always in search of new ways to show off the work they've done to improve performance of the Chrome browser, Google's Chrome team has unveiled a new JavaScript benchmark suite, called Octane.

The new suite includes all of the tests that were part of the Chocolate Factory's earlier V8 Benchmark Suite but adds a new set of benchmarks that were created from full versions of well-known web applications and libraries.

The new tests model a variety of intensive JavaScript usage scenarios, from 2D and 3D graphics rendering to in-browser code compilation. One test also times how fast the browser can load and begin using popular JavaScript libraries, including Closure and jQuery.

"A high score in the new benchmarks directly translates to better and smoother performance in similar web applications," wrote Chrome project manager Stefano Cazzulani in a post to the official Chromium blog on Tuesday.

Google engineers say the new benchmarking methods beat those of earlier JavaScript test suites, such as Kraken and SunSpider, because they test JavaScript engines against real-world code and applications, rather than using arbitrary test cases to stress specific browser features.

"Many micro-benchmarks, such as SunSpider, were written at a time when JavaScript wasn't used as extensively as a cornerstone of large, rich web applications," explains the Octane FAQ. "Therefore they tend to not measure the performance of JavaScript Engines under the demanding JavaScript environment that a modern web appplication creates."

The Octane suite gives an individual score for each test, plus a final score that is the geometric mean of all of the individual scores. Google says that it will keep adding new tests to the suite over time, so the final scores from one version of the suite will not be directly comparable to scores from other versions.

The individual tests will never change once released, however, which means individual test scores can be compared to results from different versions of the suite.

Just for kicks, your Reg correspondent took the new suite out for a spin with a few browsers. Not surprisingly, perhaps, Google's own app beat the competition, with Chrome 21 earning a final score 8653, while Firefox 14 only scored 5790 on the same hardware. The scores were based on an average of three runs.

Benchmarking Internet Explorer proved more problematic, however. We didn't have a Windows 8 machine with comparable specs handy, and it was no surprise that the IE 10 Preview struggled to complete the suite on a Samsung Slate fondleslab, scoring only 1661.

On the other hand, IE 9 wouldn't run the suite at all. In fact, it wouldn't even render its homepage properly. Are we surprised? ®


Other stories you might like

  • Makers of ad blockers and browser privacy extensions fear the end is near
    Overhaul of Chrome add-ons set for January, Google says it's for all our own good

    Special report Seven months from now, assuming all goes as planned, Google Chrome will drop support for its legacy extension platform, known as Manifest v2 (Mv2). This is significant if you use a browser extension to, for instance, filter out certain kinds of content and safeguard your privacy.

    Google's Chrome Web Store is supposed to stop accepting Mv2 extension submissions sometime this month. As of January 2023, Chrome will stop running extensions created using Mv2, with limited exceptions for enterprise versions of Chrome operating under corporate policy. And by June 2023, even enterprise versions of Chrome will prevent Mv2 extensions from running.

    The anticipated result will be fewer extensions and less innovation, according to several extension developers.

    Continue reading
  • Google has more reasons why it doesn't like antitrust law that affects Google
    It'll ruin Gmail, claims web ads giant

    Google has a fresh list of reasons why it opposes tech antitrust legislation making its way through Congress but, like others who've expressed discontent, the ad giant's complaints leave out mention of portions of the proposed law that address said gripes.

    The law bill in question is S.2992, the Senate version of the American Innovation and Choice Online Act (AICOA), which is closer than ever to getting votes in the House and Senate, which could see it advanced to President Biden's desk.

    AICOA prohibits tech companies above a certain size from favoring their own products and services over their competitors. It applies to businesses considered "critical trading partners," meaning the company controls access to a platform through which business users reach their customers. Google, Apple, Amazon, and Meta in one way or another seemingly fall under the scope of this US legislation. 

    Continue reading
  • UK competition watchdog seeks to make mobile browsers, cloud gaming and payments more competitive
    Investigation could help end WebKit monoculture on iOS devices

    The United Kingdom's Competition and Markets Authority (CMA) on Friday said it intends to launch an investigation of Apple's and Google's market power with respect to mobile browsers and cloud gaming, and to take enforcement action against Google for its app store payment practices.

    "When it comes to how people use mobile phones, Apple and Google hold all the cards," said Andrea Coscelli, Chief Executive of the CMA, in a statement. "As good as many of their services and products are, their strong grip on mobile ecosystems allows them to shut out competitors, holding back the British tech sector and limiting choice."

    The decision to open a formal investigation follows the CMA's year-long study of the mobile ecosystem. The competition watchdog's findings have been published in a report that concludes Apple and Google have a duopoly that limits competition.

    Continue reading

Biting the hand that feeds IT © 1998–2022