Sophos Cloud Optix
In late 2011, we published our analysis of a bunch of USB keys we’d bought at a lost property auction.
We took $400 (about £260) to a public transport utility’s annual auction and came home with 57 USB keys containing 4400 files.
We didn’t find any Wikileaks-type information – there were no criminal plots, no rush orders for F-35 strike fighters, and no intelligence data from the diplomatic community.
But we did get several surprises, which we wrote about at the time:
* 66% of the keys had one or more malware infections.
* Many of the keys contained personal and work-related files.
* Not a single one of the 4400 files was encrypted.
Our report quickly put the data protection cat amongst the pigeons, with two sides emerging in the debate.
The Government Should Do Something About It camp castigated the transport operator (Railcorp New South Wales) for selling the keys in the first place. As @KineticPearl commented on Naked Security:
What I find interesting is that [RailCorp] are not wiping the keys prior to sale. In this case it made for a good story, but a better one would be that you found absolutely nothing at all due to the diligence of the authority. I feel that the seller should be responsible for preventing data leaking, even if it is not directly their data, after all they are profiting from selling the device.
Even though I’m a strong advocate for privacy (and a staunch believer that opt-out simply isn’t good enough), I ended up in the Government Isn’t There To Nanny-State You camp, and argued the opposite:
I have thought long and hard about this. I don't think that RailCorp should be obliged to wipe the data, in much the same way that I don't think that ISPs should be obliged to watch your internet traffic and block pirated stuff. It's not data which RailCorp collected for its own use, after all
Unsurprisingly, the debate quickly drew in the New South Wales Office of the Privacy Commissioner, the body which oversees how personal information is held and used by the State’s government and public service.
We were delighted to get involved in the Privacy Commissioner’s investigation, and had some fun at SophosLabs showing the investigation team how we automated the retrieval, recovery and reporting of the data on the keys we bought.
In particular, we were able to show that the recovery process, though lengthy (USB keys tend to be quite slow, and we loaded and stored every byte from 50 of the keys), could be completely automated, apart from the insertion and removal of each key in turn.
Cleaners, for example, could acquire USB data dumps whilst working their way through an office building overnight – without attracting attention by losing time on their regular job.
We were also able to recover files from two apparently-wiped test devices brought in by the investigation team. Even USB keys that most people would consider safely blanked out and suitable for re-use may still contain critical information.
The Privacy Commissioner’s report has recently been published.
(Even if you’re usually afraid of government reports, give this one a read. At just three pages it is concise, clear and uncompromising – but without being cynical or judgmental.)
The bottom line of the report – literally and figuratively – is that @KineticPearl was right, and I was wrong.
Railcorp made a proactive decision to destroy lost USB keys in future, rather than to try to wipe them and sell them, and the Privacy Commission was pleased:
The Privacy Commissioner considers that RailCorp's assessment of the risk to the privacy of individuals is correct and that the decision to cease auctioning USBs is the most reasonable outcome.
The Privacy Commissioner commends RailCorp's decision, made without waiting for the completion of this inquiry.
This begs the question, “What to do with your USB keys?”
Wiping USB keys that are being retired from service might not have the purgative result you want. Destroying them instead is an effective and simple alternative.
But how do you protect yourself from leaking data on USB keys which get lost? Or on keys which are transferred between users, departments and even companies?
One answer: only ever write encrypted data to your USB keys.
(Yes. We have crypto products to help you do just that. I’m not suggesting you use these products simply because we have them – it’s the other way around. We have the products because we think they’re the right way to solve this problem.)
The most shocking thing in our original research was not the high prevalence of malware, nor the fact that the keys got sold in the first place, nor that USB keys are so easy to lose.
The most shocking thing was that not one file on any of the keys we bought was encrypted – even those files which contained personally identifiable information or proprietary information from work.
Encrypt everything and you never have to worry about the stuff you didn’t encrypt!
–
PS. Why not try our free encryption tool?
It’s an easy way to save and share files securely, whether you use removable devices or cloud-based internet services.
(Direct download; no registration required. Sorry, Mac users: this one’s for Windows only.)
It’s absolutely free to throw them into the fire. Problem solved.
If you lose it & no longer have it,how can you throw it into the fire?
"Sorry, Mac users: this one's for Windows only."
That's probably one of the reasons why data encryption on USB keys is not more common. It often requires that some form of encryption software be installed on *all* machines that the USB drive is to be used on. Further, that software may not be cross-platform. It may work on Windows, but not on Mac or Linux.
There are many organizations which forbid employees or guests from installing software on company computers. Some even go further and prevent any software that is not on the "white list" from running, even if it's a portable app installed on the USB drive itself.
Because of these limitations, it is much less of a hassle to leave the data unencrypted, despite the added risk if the device is lost or stolen.
I guess that using password-protected documents (like MS Office files, or ZIP and RAR archives) would be a compromise.
Any organization that goes to the trouble of locking down every application should also be smart enough to provide encryption tools to employees and mandate that they be used for all sensitive data.
As pointed out by Russ, Truecrypt is cross-platform. It's also free.
Senior management is the problem, as the ‘do as I say rather than do as I do’ rule applies. This is rather ironic they are the target of ‘steal to order’ industrial espionage. Culture change needed.
Truecrypt encryption runs everywhere.
Even the school I attended required students and staff to encrypt their USB drive. No encryption = no USB drive. They enforced this with domain policy.
I am amazed that companies and individuals with much more confidential data than school work aren't doing more about this.
What kind of wiping have you tested to be insufficient to actually prevent reconstruction of the data? I usually overwrite the whole device with zeros using the following command on linux:
dd if=/dev/zero of=/dev/disk/by-id/usb-XXX bs=1M;sync
This should actually prevent all data reconstructions attempts via the standard USB Storage protocol. However, it may still be possible to get some data back by desoldering and reading out the flash chips (or maybe some undocumented debugging commands of the usb flash controller chip).
I consider dd satisfactory. (I wiped the USB keys in the report that way after I was finished 🙂
There are a few things to bear in mind:
* The sector-level writes you make from 0 to maxlen-1 using dd don't necessarily go directly to sectors 0 to maxlen-1 in physical flash. There might be wear levelling, bad sector remapping and more to make writes go to sectors other than you expected.
(On a cheap USB key, my assumption is that there aren't an awful lot of smarts in the firmware, and that dd across the entire '"officially-admitted" sector surface is sufficient.)
* It's worth verifying the write afterwards, if only by reading back some of the sectors rather than the entire device.
(One trick I've used when wiping hard disks after malware analysis courses is a script which satisfies me that "she'll be right" before I start. I read N randomly-spread sectors, where N is a smallish integer. I modify the sectors in some way so that they aren't all zeros, write them back, and read them in again. This confirms I can read and write somewhat arbitrarily on the disk. Then I zero everything, and once again read back the N original "sacrificial sectors" to check they have now settled on zero. I also write one sector at a time just past the claimed end of the disk and verify that the writes fail, to satisfy myself that the device's reported size at least seems to cover the whole surface.)
* You can't wipe a USB key you just lost 🙂
If your truly worried about data security the best method is hardware encryption and a good encryption key. Get the pass wrong too many times and the encryption key is dropped from the file making the data useless even if you had the right key. Don't wast your time with 256bit software encryption, it's like leaving a key to the front door under the mat, the door might be locked but anyone why half a brain can still get in.
Errr…I don't buy your suggestion that using 256-bit software encryption (by which let's assume you mean AES-256) is "like leaving a key to the front door under the mat."
(A decent hardware-based encryption key will, in any case, use AES-256 – or perhaps AES-128, but let's treat that as more similar than different. If it doesn't, you probably ought to be asking, "Why not?". And when we say "hardware-based", it's more like "firmware-based" or "flash-based" – there's a CPU and a miniOS and some driver software on the device itself.)
"Auto-destruct after N failures" is a nice feature, provided it doesn't lead you to choose a dodgy password (one which could perhaps be guessed within N tries) precisely because it feels less risky to choose badly.
Of course, a punishing problem with crypto-on-the-key USB devices is that they're _much_ pricier than regular USB keys – and you can't just pop out to OfficeWorks with $5 and buy a new one when you're on the road and need an extra gig or three of storage.
Software-based crypto with a decently-chosen password is perfectly satisfactory. It also has the benefit that you can use it with all your data storage options, from USB through hard disks to the cloud…
Have to disagree with you about the value of hardware-based encrypted USB keys. Hands down the best gift I ever got for my dad (Kanguru Defender Elite). If you use a password manager, there's no worry about losing it or getting it wrong, so you can make a nice secure passphrase.
They are more expensive (the one I got was $40 for 8GB), but they also tend to be higher quality. A lot of them are sealed with a resin to make them tamper-proof. That also makes them water-proof (poor thing has seen so many puddles; even got a taste of fine brandy once and still works great). Kanguru will also do custom engraving on them, so it has a higher chance of coming back to you if it gets lost (their drives also have datalocks on them to help avoid malware).
Usage probably dictates whether "hardware" or software is better in this case. If you have a lot of small files like documents, and need to be able to view them on a lot of different platforms or can't count on having admin rights, and aren't that worried about the cost (or losing the files is much, much more costly), then hardware encryption is better. But if you have a lot of really big files, you don't move platforms a lot or can count on admin rights, or your cost-conscious, then software encryption is better. Plenty of room for both to co-exist.
Hardware based encryption can be defeated with a hardware-based attack. For the hardware encryption to be effective against a determined attacker, the device must be constructed in a way that it cannot be disassembled without destroying the memory. Because such a design construction is much more expensive to manufacture, most hardware-encrypted devices will not be made this way, which means they can be pulled apart and desoldered to defeat the hardware encryption.
If you get a device that is "hardened" to prevent this, it will cost much more for the device.
I wonder if two entirely different programs using GPG will encrypt and decrypt the same file. That would solve the problem of making the data recoverable on Macs, Windows PCs and Linux computers, both at home and the workplace.
GPG is supposed to work across platforms. (It wouldn't be much use otherwise 🙂
I have used it (in training coursework to encrypt malware) to share and decrypt files between Linux, Windows and Mac. For straight symmetric crypto (where sender and recipient pre-share a secret key, rather than using a pair of public-private key-pairs), use a command like this:
$ gpg -c –cipher-algo AES256 filename
There's also LUKS for Linux and its Windows equivalent, Free OTFE, as open source methods of on the fly encryption.
Linux also has the ability to securely wipe the drive without destroying it: either by overwriting everything on a byte-by-byte basis with pseudorandom data (quicker, but less secure) or completely random data (the most secure, but takes forever).
Overwriting drives sector-by-sector can be done with "dd" on all platforms, including Windows, using a suitable port of "dd". You just need to know the device name which corresponds to the physical device (not a partition like /dev/sdc1 on Linux or E:\ on Windows).
On OS X, you also have the rather neat command:
$ diskutil zeroDisk devicename
This is like dd if=/dev/zero … but shows a simple but effective text-mode progress bar.
On Linux, even reading from /dev/urandom (the not-so-seriously-hardware-random stream) is absurdly slow. I'd suggest that just writing zeros is entirely satisfactory, given the caveats I mentioned to @Jakob above…
Unfortunately with flash devices, this is not necessarily a valid solution. USB devices have a controller which provides translation between the linear addressing of the flash memory chips and the sector based access of the PCs OS.
Also, each device can have as much as 25% extra memory over and above the stated capacity. This allows the controller replace bad locations with spare ones. The controller also handles wear leveling, i.e. caters for the fact that flash memories wear out with use. The controller maintains an internal mapping of the logical sector being addressed by the OS and the physical memory it is mapped to. This physical location is cycled so all the memory locations get used equally.
Now, in terms of wiping a USB flash drive by overwriting it with zeros, you would not be wiping all of it. If you write a location once, the next time you write it you would write to the same logical location but this would be at a different physical location.
One way to do it would be to create a large file the same size as the drive and write it three or four times. The wear leveling process should then do the job of overwriting all of the physical memory include the spare bits.
Unfortunately, even this may not work as some USB flash controllers use compression. So, if the file you use to overwrite it is just zero padded, the actual bytes written will be tiny and will not therefore overwrite much at all. This could be fixed by creating a file with random data and pre-compressing it.
I should point out that there is no logical way for the OS to directly access the physical memory on the device so unerased data could only be accessed by physically removing the chip and accessing its contents directly via special hardware. Such devices do exist. Unless there is something of serious value on the device, of course nobody would be likely to go to those lengths.
Therefore the easiest way really to sanitize a USB flash drive given that they are so cheap is to employ what in my part of the world in known as a "Birmingham Screwdriver" aka a hammer!
For further info on sanitization of USB flash devices take a look at the following link:-
http://static.usenix.org/events/fast11/tech/full_…
It is a bit of an eye opener.
On Linux, You can speed up /dev/random (and urandom) by installing randomsound.
It uses noise from your microphone input the seed the random number generator to give you plenty of entropy. For best results don't connect a microphone, and turn the gain up to maximum so it gets lots of noise from the electronics.
On a headless server of mine I have connected a cheap usb sound card (the size of a usb key) for that sole purpose, and it works nicely.
If you need more entropy, then google for "entropy key" HW random number generator, 32K/second for about $40
As clear from my comments quoted I think its important to ensure personal information is protected even when found by an corporation is protected. After all if that information pertained to that corporation, and then was on-sold there would be hell to pay.
If that means the destruction of the device then so be it. Hopefully the destroyed keys are then recycled. Good call by the Privacy Commissioner IMHO, and great to see the story followed up. Also, well done to Sophos for bringing the whole problem to light in the first place.
USB keys are insecure and unstable… I've got 8GB of free space on Dropbox that does everything I used USB keys for and more, securely and quickly. Various USB keys now sitting uselessly by the computer. Problem solved.
That's fine if you have relatively small files and a broadband Internet connection everywhere that you need your files, but what if you are in a place with a slow (or no) Internet connection? 8GB of data over a 1.5 Mbps DS1/T1 connection = 12.1 hours to download. 🙁
Just go out and buy an IronKey, high security with only ten chances to get your password right. If you don't it destroys it's self along with the date. Problem solved!
Is there a theoretical speed advantage in writing all the memory locations to 0xFF? I ask because this is the erased state of most flash memory when bought new, and when a sector of flash is programmed with anything else, it has first to be erased back to all 0xFF, and then programmed with the new information. If no reprogramming is necessary, then the chip logic has only to do the erase part of the operation.
What makes thing worse is that my sandisk even came with free encryption software – on it!!! Ok it may not be the best and it is a pain to use – but it's pre loaded and better than nothing!!
There are several FREE apps that will overwrite anything on the USBs. up to the Guttman algorirthm (35 passes). If that doesn't do it, hammer that sucker flat. put it in a blender on puree overnight, burn it with an oxy/acetelene torch then bury what's left at a crossroads at midnight on a full moon with a stake through it's heart.
Maybe cease the opportunity to get free marketing and push your endpoint encryption product. Oh that’s right it’s already been done.
Have a heart, Sir/Madam!
Are you saying we're not allowed to link to our own products on our own website, at least once in a while? Seems a bit harsh…
Snert, made my day!
Interesting report and interesting results. Thank you guys!
A couple of odd questions:
Did I correctly understand from the report that the transport authority's practice was to plug the keys into their computers to look for info so they could return the key to the owner?
That is a "nice" thing to do (incredibly stupid, but nice!) but what about the security risks to their systems and the possible retention of the owners data like say racey photos or other confidential info on the authorities' or their employees computers?
Well, now that they have a USB key policy in place, what about other eGizmos? Even cameras have on board memory and memory cards too, forget about devices like The universe of iGizmos, the other assorted smart phones, and other portable computers…. Are they doing to be destroying all of those devices too?
And what exactly is their "secure" method of destruction?
Best wishes from Atlanta, GA, USA and Thank you to you and the whole Sophos Team once again!
This post was originally added to one of the previous threads but as there are several contributors referring to sanitation software utilities, I have reproduced it as a new comment with a few amendments.
Firstly, there is no point doing a 35 pass Gutmann run on a flash drive. This technique was originally meant for older conventional hard disks that had inaccurate head positioning. The Gutmann method takes account of this ensuring that the full width of the magnetic track on the disk surface is erased thus defeating advanced hardware based recovery systems. See the link below:-
http://pcsupport.about.com/od/termsg/g/gutmann-me…
This is not applicable in a solid state device so one pass is plenty.
However, unfortunately with flash devices, using sanitation utility software is not necessarily a valid solution anyway. USB devices have a controller which provides translation between the linear addressing of the flash memory chips and the sector based file system access of the PCs OS.
Also, each device can have as much as 25% extra memory over and above the stated capacity. This allows the controller to replace bad locations with spare ones. The controller also handles wear leveling, i.e. caters for the fact that flash memories wear out with use. The controller maintains an internal mapping of the logical sector being addressed by the OS and the physical memory it is mapped to. This physical location is cycled so all the memory locations get used equally.
Now, in terms of wiping a USB flash drive by overwriting it with a sanitation utility, you would not be wiping all of it. If you write a location once, the next time you write it you would write to the same logical location but this would be at a different physical location.
One way to do it would be to create a large file the same size as the drive and write it three or four times. The wear leveling process should then do the job of overwriting all of the physical memory including the spare bits.
Unfortunately, even this may not work as some USB flash controllers use compression. So, if the file you use to overwrite it is just zero padded, the actual bytes written will be tiny and will not therefore overwrite much at all. This could be fixed by creating a file with random data and pre-compressing it.
I should point out that there is no logical way for the OS to directly access the physical memory on the device so unerased data could only be accessed by physically removing the chip and accessing its contents directly via special hardware. Such devices do exist. Unless there is something of serious value on the device, of course nobody would be likely to go to those lengths.
Therefore, as others have suggested, the easiest way really to sanitize a USB flash drive given that they are so cheap is to employ what in my part of the world in known as a "Birmingham Screwdriver" aka a hammer!
For further info on sanitization of USB flash devices take a look at the following link:-
http://static.usenix.org/events/fast11/tech/full_…
It is a bit of an eye opener.