Sophos users woke up to mayhem on Thursday after the business-focussed antivirus firm released an update that classified itself and any other update utility as a virus.
As a result enterprise PCs running the application went haywire, generating false positives reporting SSH/Updater-B malware. Sysadmins were bombarded with automated alerts by email about the bogus problem. The issue was resolved with a functional update, issued on Wednesday evening.
For many, troubles continue because many endpoints and corporate networks hit by the false positive have been left with systems that can no longer update themselves properly because the required functionality has been consigned to quarantine.
A knowledgebase article from Sophos explaining what to do to resolve the problem can be found here.
False positives hitting antivirus updates have affected all vendors from time to time. The consequent problems are at their worst when Windows operating system files are falsely classified as potentially malign and quarantined, resulting in unstable or unusable Windows boxes.
Sophos's auto-immune screwup caused just as much if not more pain, judging by the large number of reader emails we've already received on the topic.
"About 9.20 this evening, every PC on my network (about 100 of them) started sending me an email every 10 minutes saying that a virus had been detected in one of the DLL files of Sophos Endpoint Security & Control," said Reg reader Iain H in an email that's typical of those we've received.
"It's obviously a nasty false positive - in that it is disabling the AV itself so (a) we're open to threats and (b) it may not be able to update itself when the new signatures are released," added the hard-hit admin.
Iain spent some time trying to get someone from Sophos support on the phone before receiving confirmation that a false positive was behind the snafu.
"It's a bit of a face-plant: classifying your own product as a virus. Still, I am sympathetic. Whilst 'friendly fire' is never a good thing, it's perhaps an inevitable accidental consequence of any long-running and intense conflict. However friendly fire normally means shooting your friends - not your own foot," he added.
Questions will inevitably arise about Sophos's testing and quality assurance process in the aftermath of the screw-up. Glenn C explains the impact of the problem.
"A fix was quickly released from Sophos but the auto-update program was quarantined and wasn't allowed to run. For a home user, a quick disabling of on-access scan, update, re-enable on-access scan and remove any items marked with the Shh/Updater-B virus from quarantine fixed the issue," he said.
"For corporate users, it was a little more problematic as it's not possible to remove items from the quarantine using the Enterprise Console. You can use the Enterprise Console to push out a new policy with either on-access disabled (not recommended) or at least excluding the Sophos auto-update folder from on-access scanning. This meant it was possible to get the latest update out to the clients - however it is still necessary to go to every single impacted system and clean out the quarantined items. Ouchie."
Customer discussion of the issue can be found here. ®