Shh/Updater-B false positive by Sophos anti-virus products

Filed Under: Featured, Uncategorized

Latest information:

Knowledge base article: http://www.sophos.com/en-us/support/knowledgebase/118311.aspx

We will continue to update the knowledge base article above with the latest advice for self-service. Please consider following our support team @SophosSupport on Twitter for updates.

Updated article below:

Some Sophos customers have reported detections today of Shh/Updater-B.

Many of these reports involve detections of Sophos's own code, but there are a number of third-party applications which are also being identified.

Sophos would like to reassure users that these are false positives and are not a malware outbreak, and apologises for any inconvenience.

False positive

If you have Live Protection enabled, you should stop seeing these detections as the files are now marked "clean" in the cloud. (Details of how to enable Live Protection can be found in this knowledgebase article).

If you do not have Live Protection enabled you will stop seeing the new detections once javab-jd.ide has been downloaded by your endpoint computers (released at Wed, 19 Sep 2012 21:32 +0000 UTC).

There is no cleanup for this detection, and you will see it quarantined unless you have your on-access policy set to move or delete detections if cleanup is not possible.

Please double check your SAV policy under cleanup; You want to ensure your secondary option (when cleanup is not available or does not work) to be set to ‘deny access’ and not delete or move. Once the detections have stopped, you can acknowledge the alerts in the Console, this way you can see who is still reporting it, and confirm it is trending down.

You should also check that any third-party applications that may have been erroneously detected as Shh/Updater-B are restored.

Further information:

Knowledge base article: http://www.sophos.com/en-us/support/knowledgebase/118311.aspx

The knowledge base article will be updated as appropriate. Please consider following our support team @SophosSupport on Twitter for updates, and if you have a support question use our online product support forum.

, , ,

You might like

102 Responses to Shh/Updater-B false positive by Sophos anti-virus products

  1. Ryan · 765 days ago

    You managed to push out a false positive which flags your own update utility as a trojan and quarantines it, and the solution is...wait for it....to update Sophos using the now-quarantined update binary. Well done.

    • GreyBeard · 761 days ago

      The irony of the situation is apparent - but only if you apply a certain meaning to "quarantined". With the recommended settings it has just been prevented from running and put on the "bad guys" list. For many (actually I assume: most) installations all you needed to do was to centrally (no need to perform any action on the individual endpoints) "unblock" it and it corrected "itself". And subsequently all other blocked updaters were allowed to run again as well.

  2. Sootie · 765 days ago

    That was an interesting morning, more so for you guys I would bet

  3. sean · 765 days ago

    You need more information for those of us who did not see the issues until after the update file was deleted. You state we need to change our settings to quarantine but if it is "too late" what is the fix?

  4. Brad · 765 days ago

    If your policy is set to delete or move this is what you will see are a ton of auto-updater binaries completely gone.

    Here is an example of a common one for any system with Flash installed:

    Infected file "C:WINDOWSsystem32MacromedFlashFlashPlayerUpdateService.exe" has been deleted.

    I've seen this for a number of other applications on systems as well.

    For us, we are a healthcare facility and this Shh/Updater-B false positive detected a DLL file in our Electronic Health Record (EHR) system and deleted it. Without this DLL the application would not open and the application crashed open execution on hundreds of computers in the clinic including the servers. This has been a huge threat to patient safety and extensive downtime for us.

    We will be cleaning up for a long time after this. Our Security/AV vendor has created a problem that no virus/trojan/spyware/malware has ever done to a company I have worked for in my 16 years of working in IT.

  5. a_v · 765 days ago

    Our's deleted 2 of its own update components. I think it's pretty funny actually.

  6. Narendra Rai · 765 days ago

    All the Sophos icons from individual workstations are not shown in the system tray and EC is reporting Shh/Updater-B, is this normal? Tried above suggestions but did not work in my case.

  7. Nate H · 765 days ago

    What do we do if in the Enterprise Console we had the Cleanup tab set to 'Deny access and move to default location'? The false positive has flagged C:program files(x86)Sophosautoupdatealsvc.exe and moved this file to the default location and I am unable to push out updates to clients. We have 7,000 computers managed by the Enterprise Console.

  8. Graham Cluley · 765 days ago

    Folks - if you require online technical support over this issue then please visit our online support forum at http://bit.ly/sophossupportforum where our online support engineers and fellow customers can assist.

    I'm afraid we can't provide effective tech support via Naked Security. Thanks for your understanding.

    • Jezza · 765 days ago

      Graham, what about your phone support, one number constanly has the engaged tone and the other I've been on hold for 45 minutes.

      If this was a virus outbreak how would Sophos deal with the influx????

      • Graham Cluley · 765 days ago

        I suspect Sophos has received more phone calls today than we've ever had in our entire history.

        Yes, more than the Love Bug, Conficker, you name it...

        If you have had a problem getting hold of us via the phone - I can only apologise. It's just the sheer number of calls we are receiving.

        Our teams are working around the clock, dealing with each caller and assisting them with any problems they are experiencing. I know it's very frustrating, and we are very sorry about what has happened. I would recommend keeping an eye on the knowledgebase article for updates, as well as our online product support forum at http://bit.ly/sophossupportforum

        Once again, we're very sorry about what has occurred, and are working very hard to resolve the issue for those customers who are still affected.

    • Steve · 764 days ago

      Good luck trying to get through to technical support. We are going on day two of this debacle and haven't gotten a call back yet.

  9. Chad · 765 days ago

    Can you at least post a resolution to the probelm your company created that actually works. I spent almost 4 hours on hold today without speaking to anyone.

  10. Phillip Mui · 765 days ago

    How can we force update workstation from SEC?

    • rodmacpherson · 765 days ago

      You can select the machines in EC, right click and click "Update Computers Now"

      If you are really stuck with some machines that continually re-detect the updater as malware you can delete the IDE on those workstations and restart the service. I have a CMD script for that here: http://rodstech.blogspot.ca/2012/09/sophos-false-...

      If you had set Sophos to move or delete the file instead of blocking access as recommended, then you might not have an updater on the workstations anymore to update them. In this case you can "Protect" them again to roll out a fresh install from the EC.

  11. ShakeOfTheHead · 765 days ago

    Hopeless, we've got so many systems with no anti-virus at the moment because of this advice, and it really hasn't fixed the problems.

  12. James · 765 days ago

    Hi,

    Have followed the last step as our Endpoint Clients are still flagging alerts, i dont see how disabling 'on access scanning' can possibly work or the logic behind this, i've done this through Enterprise console and now at even more risk changing this policy. This is starting to get frustrating when the fix doesnt work yet it doesnt make any logical sense?

  13. Steve · 765 days ago

    Thsi does not resolve the issue for people that have their policies to move/delete.
    basically from what I am reading they are stuffed and have what appears to be a massive project on their hands.
    Not good enough.
    Sophos should be fixing this for everyone, inclusive of rebuilding machines from scratch if that's what they have to do.
    Nothing less than that is acceptable.

  14. Frankie · 765 days ago

    I get really angry. I get busy for whole this morning for solving this problem and still not yet fixed all endpoint until now. It is unbelievable that Sophos could release an UNTEST SUBSCRIPTION to the public.

  15. John · 765 days ago

    Wow, way to go Sophos, my gf ended up deleting a lot of her "infected" files because of this and now her computer is acting strange. FAIL.

    • @hiramiyaa · 765 days ago

      There is a niggling part of me wondering why your 'gf' has Sophos on her computer, but I guess that's a different story..

      • rodmacpherson · 765 days ago

        Maybe his 'gf' works for or owns a company that subscribes to Sophos?

  16. Hugo · 765 days ago

    hi Guys, I am not very experienced with Sophos, so please help me with this. Do I need to recover the files which have been put in the quarantine and if so, how do I do that.

  17. Yahoo · 765 days ago

    How stupid can one company be? I guess million of users will no throw out there Sophos software....

    • Rob · 760 days ago

      Who would they turn to instead? Sure this is the worst mistake Sophos has ever made - in 25 years! Other AV companies have much worse track records.

      Maybe you would be better of without any endpoint security at all? But if so I wouldnt recommend ever connecting to the internet or allowing USB drives and MP3 players within 100 yards of your pcs...

      Seriously, I would say judge Sophos not by this mistake, but by how they respond to it. Seems to me they have been doing exactly what other posts on here are saying they should do - working round the clock to fix this for all afftected customers. constant updates to the knowledgebase article, resolved for the vast majority of customers within a few hours, and working hard for days to write scripts for those for whom the first fixes didnt work. I take my hat off to them!

  18. Peter Vink · 765 days ago

    After following the steps, computers still won't update and some endpoints are trying to update constantly.
    I get a message from "Sophos AutoUpdate" saying "Internal Error 2324. 6, C:Program FilesSophosAutoUpdateps.crl

    • LozT · 764 days ago

      If you rename ps.crl (which is 0 bytes) to ps.crl.old, then run 'ALUpdate.exe' from 'C:Program FilesSophosAutoUpdate', this will fix the error you're getting, re-enable the systray icon, re-add the 'Updating' section (which will also be missing prior to running this) into the Sophos Endpoint Security and Control, and will re-create ps.crl with the same properties/behaviours as before renaming the original...or that's worked for me anyway...

  19. Hank Anrold · 765 days ago

    We were hit with this. I've disabled "on-Access" until it's cleared up, but what do I do with the end ponts that are alread qarantined? In teh quarantine, the only options are "move" and "delete"..
     
    TIA,
    Hank Arnold (MVP)

    • Hank Anrold (MVP) · 764 days ago

      In our AD domain, the end user is very tightly controlled. They can't inastall *ANYTHING* and they only have "Read" access to any of the folders affected. This resulted in the files remaining in the original folders, but blocked from running.

      We had about 35 machines affected. I followed the steps recommended by Sophos to delete the bad file and re-sync the SUC. I then pushed the update out to the end points.

      Once I verified that my computer had received the updates, I cleared the quarantine list (no delete or move). Seeing that my workstation wasn't reporting any more viruses, I turned the On-Access scanning and pushed the updates out again.

      I ended up with only 3 end points that needed to have the client protection re-installed.

      Hope this helps.
      Hank Arnold (MVP)

  20. Ja · 765 days ago

     @Hank Anrold If you have disabled "on-access", try to run in command line "C:Program FilesSophosAutoUpdateALUpdate.exe" -ManualUpdate". It should update endpoints. First of all, update SUM!

  21. ChrisUK · 765 days ago

     @Hank Anrold I think what Hank is trying to say is will it clear the reported files from quarantine? 
     
    I'm currently dealing with this issue too... Grrrr!!!

    • Hank Anrold (MVP) · 764 days ago

      No, it won't clear the quarantine (at least not for me). Once I refreshed the SUC and pushed out the updates, the blocking of the files was turned off. However, on each end point the user has to "clear" the list in teh quarantine and I had to acknowledge teh warning in the SEC.

      I was fortunate in that onl 3 out of 35 affected machines had to re-install the protection. 100+ end points were unaffected since they never got the corrupted update.

  22. James Elmain · 765 days ago

    Has the comments section been changed to hide the previous comments, what's with that?

    • Craig Brand · 765 days ago

      Looks like a cover up!

      • Graham Cluley · 765 days ago

        Smirk.. no cover up.

        With appalling timing, we have had issues with the comments system on the Naked Security site as part of a switch we're planning away from IntenseDebate. Apologies to anyone whose comment may have vaporised.

        We're trying to be as open as possible about the problem, and are trying to help customers the best we can.

        • Craig Brand · 765 days ago

          I know you are Graham thanks just a gentle dig in the ribs!

        • Nigel · 763 days ago

          Graham:

          Not really on topic, but I can't help commenting on the news that NakedSecurity's planning to move away from IntenseDebate...a hopelessly primitive system (at least in its implementation here) that allows no formatted text, and no editing of one's own posts after they're submitted. Utterly barbaric. One can only hope that whatever new system NakedSecurity uses will rectify those deficiencies.

          Back on topic, I'm terribly sorry to hear about the snafu with the Shh/Updater-B false positive. If doesn't affect me, but I'm nonetheless aggrieved for the sake of those users who are affected. Not being a victim myself, I can also empathize with the folks at Sophos, whose hearts are in the right place (as the existence of NakedSecurity amply demonstrates), and who must be pulling their hair out as much as their unfortunately afflicted customers are. Best wishes for a speedy recovery to all concerned.

  23. Ray_McGhee · 765 days ago

     @ChrisUK  @Hank Anrold This is a clusterfcuk of the highest magnitude.  I have had no success with the Sophos fix.  Manually deleting the problem IDE then either forcing an update from the console or restarting the Sophos AntiVirus service has fixed some.  Don't want to have to attempt that on 400+ broken cleints.  Come on Support, give us something scripted!

  24. Graham Cluley · 765 days ago

    Nice conspiracy theory - but not quite what's going on.
     
    We are in the process of switching commenting systems from Intense Debate to LiveFyre.  Unfortunately that process has coincided with the false positive issue, and we haven't yet finished importing all the old comments into the new system.
     
    Our plan is to make old comments visible too - just may take a while.  As you can probably understand, most of our efforts right now are revolving around assisting customers with the false positive problem rather than fixing the commenting system on Naked Security.  :)
     
    Normal service will I hope be resumed as soon as possible.  In the meantime, we recommend customers with issues visit our support forum at http://bit.ly/sophossupportforum or follow our team at @SophosSupport for the latest updates and advice.
     
    Our support team isn't really set up for providing product tech support via Naked Security - best to go to our dedicated support forum instead.
     
    Hope that helps

  25. TCIT · 765 days ago

    Has anyone got the steps above to work yet? I have tried a few times with no luck! Its such a pain!

  26. Graham Cluley · 765 days ago

    Certainly lots of customers have successfully resolved the issue.  If you're having problems I would recommend visiting our product support forum at http://bit.ly/sophossupportforum
     
    We're sorry about the inconvenience, and we realise the problem is a serious one.  All we can do is apologise - right now the best assistance is going to come from our product support forum or by following our support team on twitter:  @SophosSupport

  27. TCIT · 765 days ago

     @Graham Cluley  @SophosSupport Oh sorry i'm not complaining its just a pain! Im sure I will resolve it very soon with Sophos assistance :-)

  28. Graham Cluley · 765 days ago

    Hi Ray
     
    I'm sorry to hear about the problems you're experiencing.
     
    Sophos's team is working on more ways to resolve the issue for impacted users. 
     
    I would recommend joining our product support forum at http://bit.ly/sophossupportforum and following our support team @SophosSupport on Twitter

  29. Graham Cluley · 765 days ago

    @Hank Anrold If you're still having problems please visit our product support forum at http://bit.ly/sophossupportforum and follow our team on Twitter at sophossupport 
     
    We're very sorry about what has happened and are working hard to fix the problem for our customers.

  30. pissed off · 765 days ago

    Not working for me! Why doesn't Sophos put an .exe file online for people to download and use to fix the problem? Is it too much to ask????

  31. Graham Cluley · 765 days ago

    I'm sorry the problem hasn't been fixed for you yet.
     
    We recognise the problem is very serious and are working hard to resolve it.  If you are having difficulties please visit our product support forum at http://bit.ly/sophossupportforum and follow our tech support team on Twitter at sophossupport

  32. Ray_McGhee · 765 days ago

     @Graham Cluley  @SophosSupport Thanks Graham.  I already follow you guys on FB and twitter.  Being a corporate Sophos customer for 12+ years, I am a tad concerned the only Advisory Sophos has put out with fixes (given how big an issue this is) has only recieved 1.5 out of 6 in feedback.  Given many admins will have been on this through the night, I find it pretty poor nothing further has been posted re scripting something, anything, to give us a bit of hope.

  33. Graham Cluley · 765 days ago

    @Ray_McGhee I imagine that the reason that the kbase article currently has a low score may be that people are (understandably) pretty peeved off that they've had the problem in the first place.

    Many customers have got the fix to work. We are working with customers who are still affected, and are looking into a variety of ways in which we can simplify the process for users who are still impacted.

    It's obviously important that if we post a script, for instance, that it worked properly and didn't cause more problems.

    We are taking the problem very seriously, and recognise that this has caused a highly unpleasant problem for customers.

    Our product support forum and the SophosSupport Twitter feed are the best place to be.

    Sorry again. We're trying to make the best of a bad situation.

    • Jamie Lynch · 765 days ago

      What about people like us who had a delete policy set for cleanup? Do I have to install all my clients again?

      • Craig Brand · 765 days ago

        Seems to be the only way for me... uninstall all 3 parts and run manual install from the SophosUpdate share

        • Jamie Lynch · 765 days ago

          on 2000 computers????

        • Jamie Lynch · 765 days ago

          Looks like SCCM to the rescue here. The idea I have so far for organisation policies that used a 'delete' cleanup policy is as follows;

          - SCCM script program to remove all sophos elements e.g. msiexec /x
          - Reboot
          - SCCM script to install the sophos AV program.

          I'll let you know how it goes, although I have some clients who are complaining of being unable to remove autoupdate program. Looks like a manual cleanup for that, so I might need to do some digging on this one. Any ideas?

  34. vw · 765 days ago

    Well, you really woke me up today.  I thought my worst nightmare had come true.  I am not amused.

  35. ShaneP · 765 days ago

    They may have been false positives, but this sure did feel like a malware cleanup!

    At this stage I'm thinking that we might need to "roll our own" solution for clearing the quarantine of all our endpoint machines.

    In all honesty, a question I have seen pop up a few times over this, yet have not seen answered, did updates not pass through even a basic QA test before being published?

    I could understand some obscure 3rd party app triggering a false positive, but Sophos triggering against itself? Surely that would have been a 100% guaranteed find in the most basic QA testing!

  36. Jamie · 765 days ago

    Earliest I've been up in years. Any chance of a new mug to compensate? Oh, and a stress ball...

  37. Nick · 765 days ago

    Where is the support article now? When I open it all it says is to contact support for more information??

  38. P@ssed off user · 765 days ago

    I am trying my best to be constructive here....

    No not possible...

    Sort it out Sophos...

    Oh, and thanks for the stress!

    • Graham Cluley · 765 days ago

      Sorry for the stress. We've got some smart guys and gals here who haven't been having the best time either.

      We know this has been extremely disruptive for many of our customers, and are working hard to fix things. Please accept our apologies.

  39. Craig Brand · 765 days ago

    I'm not looking forward to the daily scan at 4pm!

  40. @hiramiyaa · 765 days ago

    By the sounds of it, a few of the Sophos Devs were having too much fun with yesterday's XKCD comic (xkcd .com / 1110) to test it properly!
    And a few IT Admins here should go check that out too, lighten up and get a better perspective.. Sophos are allowed to make ONE mistake in such a long time, it's not like your AV protection is completely gone - and to update you only need to use the enterprise console to reinstall; two clicks and an authentication prompt? Not hard. (Though maybe a few more clicks if you're security conscious)

    Personally, I see this as a good thing. Sophos is cleary doing it's job REALLY well, since if the HIPS protection is catching it's own update application, malware won't stand a chance.

    • James · 765 days ago

      "Sophos is cleary doing it's job REALLY well, since if the HIPS protection is catching it's own update application, malware won't stand a chance. "

      Isn't that like saying that if the police arrest innocent people, criminals won't stand a chance?

    • Steve · 764 days ago

      How can it be a good thing?

      My Enterprise wont puch out the client anymore to infected machines.

  41. Bill · 765 days ago

    Thanks for putting this info on the website. Made our job much easier last night when first noticed. Also -- Good Job in getting it fixed so quickly. Thanks, Guys (and gals)

    • Graham Cluley · 765 days ago

      I'm glad we were able to help.

      There have been a lot of staff in our offices around the world who have been working hard on helping customers with this issue. Of course, the scale of the problem has meant that we haven't been able to help all customers as quickly as we would like. That's disheartening for us, but even worse for customers who may feel that they have been left in the lurch.

      We're working hard to fix things the best we can for all affected customers, and would like to apologise for all the inconvenience.

  42. Edan · 765 days ago

    What appears to have solved this issue for me is recreating the update share.

    Stop sharing the SophosUpdate folder (you should know how to locate a share...) and delete it, or just rename it. Don't worry about it telling you that some files are open.

    Recreate this folder and share it. Don't forget to give it the exact NTFS permissions that the original one had. Once it's ready, run Update Now on the Enterprise Console.

    **** If you've disabled On-Access scanning, don't forget to re-enable it. ****

    Edan

  43. Nephas Tembo · 765 days ago

    Goodness me!!!!,
    We really have a project under our hands here. I suspect our IT manager has opted for the delete option because all the machines are flagging up like mad!!! its really pain even when after learning its just a false positive

    Meanwhile lets see what the product support forum has for us.

  44. Damien · 765 days ago

    This fiasco has also broken Sage MicroPay and Sage accounts on our customer sites. Payroll for these customers has been severely disrupted as a result. The only resolution we have so far for this is a reinstallation of the affected applications. The suggested fix as posted here to date does nothing but attempt to repair the Sophos environment. The knock on effect for other applications could be huge.

    • Graham Cluley · 765 days ago

      We're deeply sorry about the disruption and upset that has been caused by this issue.

      Our team is working hard on a list of other applications that could have been affected by the false positive, as well as tools to fix systems at customer sites that are still experiencing problems.

      I would recommend keeping an eye on the knowledgebase article (which is being updated with more information) and our online support forum at http://bit.ly/sophossupportforum

      Please accept our since apologies.

  45. Lance Lavery · 765 days ago

    Lighten up people.. This is not half as bad as all the McAfee screwups...LOL

  46. Shane · 765 days ago

    I'm asking again about Quality Assurance, because the hundreds of corporate lawyers I am responsible to will likely be wanting a post mortem from me, as to what happened and why and what is being done to prevent a similar situation.

    How did this happen? Does Sophos not have an internal QA procedure to catch issues like this before they are published to customers at risk of not only malware, but the disasters that can come from false positives with AV software?

    We pay money for AV software to avoid loss.

    This event was minor in comparison to how bad a false positive can be, however the fact that Sophos released an update that cause a false positive against Sophos own software, suggests to me that the QA procedures in use at Sophos, if any, are highly suspect.

    It concerns me that there is high potential for a false positive of the worst consequences, such as detection of a critical OS or major application component that brings our business to a halt for hours or even days.

    I think many customers would want to know how and why this happened and what will be done to prevent something like this or worse in the future?

    • Graham Cluley · 765 days ago

      We obviously do have extensive quality assurance processes and teams inside Sophos.

      We will be conducting a thorough investigation into what went so clearly wrong, and we'll be communicating more about the background to the incident in the coming days. But right now our priority is on fixing the systems of those customers who have been affected.

      I know that's frustrating for some, and that many of you will have questions regarding the "Why" of this incident - but right now the most important thing is for us to help those who need it.

      • Shane · 765 days ago

        That's understandable and appreciated Graham. Thank you.

        We look forward to both.

    • Cindy · 764 days ago

      Has it crossed anyone's mind that Sophos might have been the victim of a cyber attack or inside sabatoge? This to me is a more plausible explanation than the Sophos I have known and respected over the years making such a colossal mistake. The release of this "mistake" during the time of a zero-day exploit is also suspect.

  47. LOL · 765 days ago

    As a quiet voice of reason, I'd like to point out that the antivirus element of Sophos is still fully functional, so PCs are not significantly at risk - all that is missing is the updating functionality. As such, there is time to sit back and think about what needs to happen.

    Yes it's inconvenient, but not as bad as some other vendors have achieved.

  48. Guest · 765 days ago

    As a veteran of the IT security & AV industry (not linked to Sophos in anyway), I believe that while this is a very unfortunate incident, you guys shouldn't be too harsh on Sophos. The definitions weren't properly tested, correct.

    But you should realize that 1) false alerts are, to some extent, unavoidable 2) you can't have both a very quick response to new threats and extensively tested definitions. 3) a long and extensive test would not be foolproof by the time it is completed given the number of files & libraries other vendors would have updated during your test.

    I've seen users on the board who wanted a full solution to this issue in less than 20 mins: that's the best recipe for further problems because they assume it would be extensively tested while it obviously can't.

    Sure, there are lessons in QC for everyone here. Meanwhile, Sophos has my sincere sympathy as I know no one is immune from such unfortunate incidents.

    • Geoff · 764 days ago

      What sort of accountability do you suggest? ... this was not unavoidable ... it was sloppiness epitomized.

    • Not my guest I hope · 764 days ago

      Guest - as a "veteran" of IT security I certainly hope you don't work for my company. Of course false alerts are unavoidable - you cannot test an update against every known application/file in existence. As an absolute bare minimum though you ensure the false positives aren't your own products files and executables. That's a simple install and scan on a test PC.

      Given Sophos and Naked Security get on the high horse about security on a regular basis, this is exceptionally embarrassing. And it doesn't matter how rare this is - the Sophos guys should know by now that it only takes one slip-up in the security field to do serious damage to a company. Could this be the one for Sophos is the big question given the impact to enterprise customers?

      • Graham Cluley · 764 days ago

        Trust me, we know that incidents like this are serious - and we don't treat them lightly.

        We obviously do have extensive quality assurance processes and teams inside Sophos - which include scanning our own product files and executables. The fact is that those tests did not highlight any problems in this instance.

        We will be conducting a thorough investigation into what went so clearly wrong, and we'll be communicating more about the background to the incident in the coming days. But right now our priority is on fixing the systems of those customers who have been impacted.

        Once again, we're very sorry about what has occurred, and are working very hard to resolve the issue for those customers who are still affected.

  49. Craig Brand · 764 days ago

    This works well

    1) Run Services.msc from Server hosting SEC

    2) Connect to the problem client for example PC1

    3) Stop the Sophos Anti-Virus service (Start | Run | Type: services.msc | Press return).
    Delete the quarantine.xml file from:

    4) File Explorer to PC1c$

    Windows XP
    C:Documents and SettingsAll UsersApplication DataSophosSophos Anti-VirusConfigQuarantine.xml.
    or
    Vista/7
    C:ProgramDataSophosSophos Anti-VirusConfigQuarantine.xml

    5) Start the Sophos Anti-Virus service.

    6) Force an update either via SEC using the 'Update Computer Now' option

    Time consuming but worked well on all the clients that had the sophos related files removed or denied access.

  50. aras · 764 days ago

    This is how I got this sorted http://www.mysysadmintips.com/windows/clients/322...

  51. NowHateSophos · 764 days ago

    I thought of a positively contructive comment - Sophos has just made Trend look like a far superior product. Thank you for shedding the light on this now, FACT! :)

    • Chris · 764 days ago

      Nowhatesophos I think that is a bit harsh as Sophos has had about 1% of the issues Trend has had with false positives in the past. One user makes a great point....Sophos looks great compared to the problems McAfee has caused users in the past. Sophos is actually doing an unbelievable job as they had a fix within 15 min and if you had Live Protection on, which you should if you knew how to use the product, you wouldn't have had any issue at all and it would have fixed itself with its next update.

  52. Joe · 764 days ago

    Wow! This hasn't affected us, so I'm not complaining. I'm looking forward to seeing, in the next couple of weeks:

    1) How this happened.

    2) What steps are being taken to make a repeat less likely.

    This will be useful in the interest of transparency, as well as a case study that will probably be widely applicable across the industry.

    I'll wait until the dust settles though. Good luck.

  53. Geoff · 764 days ago

    At our small business, the outside IT company has been here more than four hours to fix this mess. This fixes proffered took a long time to fix and to go around to each client and deal with various local issues. I sure hope there will be some credit for these costs or you won't see this company renewing any license for a product whose supplier fails to account for its damages. This event directly caused a substantial loss to the work day and unnecessary costs for its fix. Sophos is no longer a welcomed supplier for our little business.

  54. Graham · 764 days ago

    I rung Sophos support today as our College of 1,000+ computers were in chaos. Plus it also damaged the Enterprise Console as well.

    Sophos said they'd email me some info to follow as their online posting didn't work. Guess what..... they never emailed me. Then their support line was constantly engaged.

    They sent me a standard email with the same "fix info" that didn't work again at 4:50pm!

    They used to sing "Sacked in the morning! You're getting sacked in the morning!" at football matches. I may get a crowd of people to sing this outside the Sophos office. Support has been appalling!

    We have fixed 75% of our systems. But the rest look like they need manual work. And there are 3-4 levels of how badly damaged they are too.

  55. Jesse · 764 days ago

    This whole thing is a complete mess. Sophos still has no answer for how to fix thousands of machines that are messed up because the false positive Deleted the Sophos update files, etc. Not to mention, having deleted other non-Sophos files too for things like Quickbooks, etc. This will cost hundreds of wasted hours of labor to address this with larger sized customers who have a high number of clients. So far, Sophos' response has been pathetic IMHO. After making at LEAST 75 phone calls last night and today, not ONE of which has gotten through to a technical support person, I've about given up on this. There really seems to be NO sense of importance to this and I will no longer recommend Sophos to my customers as results..... pathetic.

    • Graham Cluley · 764 days ago

      We're really sorry about what has happened, and it sounds like your company has been through a lot of pain.

      We have been updating the knowledgebase article at http://www.sophos.com/en-us/support/knowledgebase... (this is an ongoing effort - we're not resting on our laurels) and scripts have been created to assist customers who have particular problems.

      Sophos is currently receiving a very high number of phone calls, and some customers have experienced great difficulties getting through to our support team. I know it's very frustrating, but please be assured that we're doing everything in our power to handle the inplux of calls and resolve customer issues as quickly as possible.

  56. Gerber · 764 days ago

    I'm so glad I moved my clients off Sophos. My old company used to sell this product because they could mark it up. What a disaster.

  57. Poonjong · 764 days ago

    Oh to live in a world where no one stuffs up, wouldn't that be nice. Surprises me just how much people bitch about this, yes, I was affected, I worked around it.

    How many of the whingers would truly complain if they were compromised and lost everything datamoneyreputation? Don't get angry at the people trying to help, get angry at the reason why we need services like this in the first place. Oh wait that is too hard.

    Bugger off and good luck on your future endeavours

  58. SupEng · 764 days ago

    Lots of chargeable time for many IT Integrators to fix this issue; 9 sites and counting - Thanks Sophos!

  59. Gary · 764 days ago

    Now spent 2 days with this....Sigh.....Sophos - Get some automated script in place, not one where we have to visit every computer! only another 500 to do?????

  60. Chadster · 764 days ago

    I have used Sophos Anti-VirusSophos Enterprise Console on networks for close to a decade. No software is ever perfect. This is a good reminder for Sophos to be ever vigilant with their QA. While I have been able to sit back and watched other companies and organization deal with major virus headaches over the years Sophos has done its job.

    Sophos employees (Nathan in particular) worked very hard to help everyone on the Sophos forum (http://community.sophos.com/t5/Sophos-Endpoint-Protection/bd-p/ESDP) along with so many other private sector professionals. It was impressive really. Scripts were being written by private sector professionals before Sophos had them! I couldn’t wait for the official Sophos script to be release so I used one listed on the forum for Sophos setting “Deny and Move”. Shortly after a similar script was release officially by Sophos. The script needs to be run on the clients but can be centrally deployed using many methods. For those of you that had Sophos set to “Delete” I feel for you but you did know what could happen if you used that setting. After running the scripts the computers just needed to be restarted and that was it.

    • Graham Cluley · 764 days ago

      Thanks for the comment Chad. I'll be sure to share it with Nathan. :)

  61. Fish · 764 days ago

    Well, I'm absolutely stuck, not a single thing has worked!!!

    • Chadster · 764 days ago

      What was your Sophos settings for "Clean up" on the clients?

  62. Topps · 763 days ago

    Restarting my machine after following the directions did not work. However manuualy disabling Sophos by right clicking or running the Stop Sophos.exe they provide. Then restarting by right clicking or using the StartSophos.exe worked like a charm.

  63. bbbbbbben · 763 days ago

    This must've been the work of a disgruntled employee.

  64. Shepherd · 762 days ago

    Sophos is the one of the best products for over a decade , its just one of their mistake which big companies like Symantec,Mcafee , Microsoft have already done . Don't forget Natwests IT blunder , this isn't as bad as that .Every one does mistake , Sophos customers should overlook this and carry on with their product. By the way i was also affected by this , i've fixed the problem and continued to use Sophos and will continue using it.

  65. John · 760 days ago

    We can all agree Sophos made a mistake. Everyone with a gripe has to consider who misconfigured Sophos to delete an infected file that could not be clean. I had very few problems since my Sophos was set to "Disable access" rather than "Delete". I went to dinner, came back.. applied the updates to Sophos and went home. If Sophos was configured properly there would few problems.

  66. SupEng · 760 days ago

    For those of you having issues, just thought I would throw my hat in the ring and detail the steps i had to take to get this working - as a Support Engineer I have resolved this at a number of sites with wildly varying setups and configs, some with OAS enabled, some without, some with files set to delete or move and some without.

    Keeping in mind that your site my differ wildly to the ones I have worked on, follow the directions at your won risk and use your head - do these instructions relate to my site?

    Now without creating the wheel

    1. work through the steps in the Sophos article, I started with the server the Sophos console is installed on - ensure agen-xuv.ide is deleted and Javab.jd.ide gets updated into the update repository (if this doesn't work, manually copy it in from a working server., refer to the article.

    2. Make sure the agen-xuv.ide file is deleted from all servers and workstations (for those of you with big sites you will need to make a script or something similar as the latest updates don't appear to remove the agen-xuv file - the Javab.jd.ide appears to download ok though when you push update from the console

    3. Once these files are updated and confirmed removed/added, acknowledge the alerts from the console, its easier to start with servers and work your way out to desktops

    4. I have also found that you will need to log into the servers and into quarantine and CLEAR the entries

    5. the console itself should start looking cleaner as you go through the affected servers/workstations and acknowledge the alerts, and make it easier to find PCs or servers where the updating files have actually been moved or quarantined - you will see that they wont be up to date

    In this instance I made a share on FP server and copied a clean copy of the AUTOUPDATE folder from a working server to the share, then on the servers that wouldn't update, copied the contents and overwrote the folder.

    6. after this you should be able to double click on the ALMon.exe and the shield will reappear in the task bar.

    If you get the resource error, (I found this was due to the files being physically removed or deleted by an overzealous user or admin) try to re-protect the affected PCs/ Servers, this should resolve the issue. if a manual push doesn't work, browse to the share where Sophos is installed and manually force it to install.

    These steps worked for me, on 11 sites now, have not had an instance of reinfection or false positive, hope this helps

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

About the author

Follow Naked Security on Twitter at @NakedSecurity, on Facebook or join us on Google Plus.