Updated Webroot's security tools went berserk today, mislabeling key Microsoft Windows system files as malicious and temporarily removing them – knackering countless PCs in the process.
Not only were people's individual copies of the antivirus suite going haywire, but also business editions and installations run by managed service providers (MSPs), meaning companies and organizations relying on the software were hit by the cockup.
Between 1200 and 1500 MST (1800 and 2100 UTC) today, Webroot's gear labeled Windows operating system data as W32.Trojan.Gen – generic-Trojan-infected files, in other words – and moved them into quarantine, rendering affected computers unstable. Files digitally signed by Microsoft were whisked away – but, luckily, not all of them, leaving enough of the OS behind to reboot and restore the quarantined resources.
We understand that all versions of Windows were affected by today's gaffe, and that a kill switch within Webroot's systems kicked in to halt the mass quarantining before any long-lasting damage was done. Webroot boasts it has 30 million users. Its software also, weirdly, misidentified Facebook and Bloomberg's websites this week as phishing sites, blocking access to them.
"We understand that this is a consumer and business issue," a Webroot rep confessed in a on its support forums. "We understand that MSPs will require a different solution. We are currently working on this universal solution now."
Suffice to say, there are a wedge of furious and confused folks on the support boards, with angry IT admins reporting thousands of endpoints going nuts.
@Webroot I seem to have installed a nasty Ransomware app. It's called Webroot. They already have my money, should I contact the FBI?— Bob Ripley (@M5_Driver) April 24, 2017
Like many IT admins today, I am dealing with a headache caused by @Webroot's signature update. I feel for their tier 1 support staff.— Eric K. ☮ (@ericemoji) April 24, 2017
Webroot, whose slogan is "smarter cybersecurity," is working on a solution for all. The timing of the file classification blunder couldn't be worse for at least one employee. Gary Hayslip was hired earlier this month as Webroot's chief information security officer, and this can't be a fun first few weeks on the job.
The biz is also looking to hire a senior software engineer for its Windows line. Based on today's kerfuffle, they might want to consider upping the headcount a bit more in this area to ensure that customers don't get hammered in the same way again, in light of February's little snafu that also left Windows users borked.
A Webroot spokesperson told The Reg: "We know how important internet security is to our customers, and the Webroot team is dedicated to resolving the issue. We will provide updates as soon as they are available." ®
Updated to add
Webroot has now released an application for its business and managed service providers to fix the issues crashing Windows machine.
"For access to the repair utility, business customers should open a ticket with Webroot support, or reply to an existing support ticket related to this issue," Mike Malloy, EVP of products and strategy at Webroot.
"Our entire Webroot team has been working around-the-clock on this repair and is implementing additional safeguards to prevent this from happening in the future. We apologize to our customers affected and appreciate their patience during this challenging issue."