Users of Webroot's endpoint security product, consumers and businesses alike, had a nasty surprise Monday when the program started flagging Windows files as malicious.
The reports quickly popped up on Twitter and continued on the Webroot community forum -- 14 pages and counting.
The company came up with a manual fix to address the issue, but many users still had problems recovering their affected systems.
The problem is what's known in the antivirus industry as a "false positive" -- a case where a clean file is flagged as malicious and is blocked or deleted.
False positive incidents can range in impact from merely annoying -- for example, when a program cannot run anymore -- to crippling, where the OS itself is affected and no longer boots.
The Webroot incident falls somewhere in the middle because it affected legitimate Windows files and sent them to quarantine.
This is somewhat unusual because antivirus firms typically build whitelists of OS files specifically to prevent false positive detections.
"A folder that is a known target for malware was incorrectly classified as bad, and Facebook was classified as a phishing site," Webroot said in an emailed statement.
"The Facebook issue was corrected, and the Webroot team is in the process of creating a comprehensive fix for the false positive issue."
The incorrect detection lasted for two hours, between 1PM and 3PM Mountain Standard Time in the U.S., and resulted in files being flagged as W32.Trojan.Gen. As suggested by the name, this is a generic detection signature intended to catch Trojan programs.
For now, Webroot has provided a solution on its community forum that involves logging into the Webroot online console and manually creating override rules for all of the erroneously blocked files.
Users then have to either wait for the endpoint client to poll the server and restore the files according to the new rules, which can take up to 24 hours, or manually trigger a forced polling for each client from the command line.
While this solution might work for home users or businesses with a relatively small number of computers, it creates problems for large environments, especially for managed services providers (MSPs).
"This is not a fix when you're an MSP," one user wrote on the forum.
"How am I supposed to do this across 3 GSMs [Webroot Global Site Manager deployments] with over 3 thousand client sites? Not good enough," said another.
One user reported that he resorted to recovering the affected files using Windows' Shadow Copy feature. Another one said that his MSP company is considering legal action because it might have to compensate its own customers for the downtime.
"We are not able to use recovery because most of the backup server cores are affected also," he said. "Some of the servers are not yet up and we look like fools."
Webroot representatives said on the company's forum that the company is working on a universal solution that will also be suitable for MSPs.
Join the CIO New Zealand group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.