Firefox armagg-add-on: Lapsed security cert kills all browser extensions, from website password managers to ad blockers

Mozillans scramble over the weekend to undo damage

On Friday, Mozilla detected a great disturbance in its Firefox browser, as if millions of voices had cried out on social media in annoyance.

Every single web extension, theme, search engine plugin, and language pack had stopped working with netizens' Firefox installations, potentially stripping any data and settings associated with them if they were subsequently removed.

For example, in a post on Hacker News, Rosser Schwarz, who works with databases, lamented how the add-on annihilation lost work stored in the Firefox container add-on.

"I did not merely 'lose some tabs'; those, I could just re-open," he said. "I lost work. That data, effort, and time are gone."

The source of the trouble was identified in a bug report as the expiration of an intermediate signing certificate, which is used to authenticate third-party Firefox add-ons, also known as extensions. With the cert's unanticipated demise, Firefox stopped allowing these add-ons to run or be installed.

For those using the Netflix add-on, there would be no video. For those using HTTPS Everywhere, there would be no enforced privacy. And the situation was similar for users of password management add-ons, content blocking add-ons, and the like.

And when people attempted to reinstall the now-dead extensions, to no avail, they potentially lost any data associated with the software.

It was "armagadd-on," as Mozillans put it.

Firefox coders dealt swiftly with the crisis, pushing out a partial fix through the company's Studies feature-testing component for Desktop users of Firefox. That didn't reach everyone so there was a second fix pushed out on Sunday – version 66.0.4 on Desktop and Android, and version 60.6.2 for ESR.

Via Twitter on Sunday, Mozilla said, "There are some issues we're still working on, but we wanted to get this release out and get your add-ons back up & running before Monday."

Among the outstanding issues is add-on signing for Firefox developers, which as of Monday morning was still being repaired.

Mozilla did offer an apology. "We are very sorry for the inconvenience caused to people who use Firefox," the company said in a blog post.

Via Twitter, Joe Hildebrand, VP of engineering for Firefox, thanked the hundreds of Mozillans who worked over the weekend to patch things up and promised a more detailed analysis of what happened.

"We owe both our users and ourselves an unflinching post-mortem to make us better in the future," he said.

The certificate stumble followed an announcement on Thursday that Mozilla, to improve add-on security, intends to disallow obfuscated add-on code, a step Google took for Chrome extensions last October as part of a broader extension platform revamp.

Mozilla said the add-on policy change had nothing to do with the certificate failure. ®