Shredding files mostly useless (review)

Bruce Schneier points that filesystems sometimes get in the way of secure file deletion.

I blogged about that six months ago (second point in that bill) after checking my understanding of the question with the developer of Inferno.

I since heard about similar stories quite a few times, either from software like filesystems or recovery systems or from hardware like Flash memory putting the content of a file in arbitrary locations. It seems to be a fairly well known fact among people who spent time on the matter.

To my mind, apart from shredding entire drives when the hardware is disposed of or goes from an user to another, companies should not waste time on shredding.

Of course, I guess Bruce Schneier would argue about encryption, rather than deletion :-)

Shredding files [4/4]: Additional details on shredding

A link to the three previous bills, please read them first:

1. Why it's useless to "shred" files, most of the time
2. Shredding empty space
3. Please shred the hard drive

Then the matters I wanted to speak about.

First, the choice of the shredding software. Given the high number of vendors for that and the increasing number of rogue security software, I advise to take only software from a well-known vendor (from its official site or from a reseller) or opensource software.
I would bet that among all the software that claim to shred files, one quarter are rogue software.

Second, the views I gave in the three previous bills only take in consideration a part of the complexity of the question. For instance, different media (RAIDed hard drives, Flash memory...) may not follow the same behaviors as hard drives. Another example: filesystems are not considered. If the setup includes a rollback system at the filesystem level, then shredding empty space might not be efficient.

Third and final: let's think practical. There is no need to buy expensive software when you don't have a need for expensive functionalities. Most of the functionalities are covered by the tools included in a basic Linux distribution (thanks ketherius (RO) for the example). There is no need to shred everything everyday if you don't handle extremely valuable information (and even then...)

EDIT 22/06/09: If you can speak French, there has been an eXCellent discussion thread on the matter on linuxfr.org.

Shredding files [3/4]: Please shred the hard drive

At this point, we don't shred files anymore and we shred the empty space when we have time and a motivation.

Now, the last important step is not to forget to destroy all of the data when the hard drive is disposed of. There is a lot of data that you must destroy, even if you destroyed your main "My documents": Internet downloaded files, drafts that you may have forgotten, saved passwords or connection parameters...

There are countless stories of companies being spied upon by use of their old hard drives. To get rid of this threat, you can use a hard drive shredder such as the one below.



OK. So, good practice is to establish a policy that forbids hard drives (including internal hard drives in the printers and xerox machines) going out before a shred. Don't donate, sell or dump an old hard drive before a shred.

Shredding files [2/4]: Shredding empty space

Once you understand that there are shadow copies of your files of value, you get it that it's useless to shred files, as is often recommended, though.

So what's next, how to ensure your files are not recovered? At this point in our reflexion, the problem is that there are confidential bytes in the "empty" space of the hard drive. So, some software provide a tool to "shred" the whole of the empty space. Here, we mean that it will browse the full length of the empty part of the disk and cover it with random patterns, to remove all chances of recovery of the previous data.



The good point is: theoretically it works. The bad point is: practically, it's unmanageable because it means using those random patterns on the size of the empty space of your hard drive. Like dozens of gigabytes. So it takes very long.

The good practice becomes: tell your top management to bring in their laptops for a good shred, before they go to a risk area (like travelling abroad to negotiate contracts). The bad practice is: present your executives with the tool and tell them to do it themselves regularly.

Why it's useless to "shred" files, most of the time

It's becoming common knowledge that a file can be recovered from the hard drive even after being removed. The basic idea is that a file = a container + a content.

When you remove the file, the operating system (whether it be Windows or Linux or else) destroys the container but keeps the content. So the actual bytes of your file remain on the hard drive. And a myriad of software (most with a shareware license) have grown to sell you the idea that by writing zeroes or random patterns over the content, it will make it unrecoverable. That's theoretically true.



The problem is that the soft only destroys what you ask it to. So if there is another copy of the file, that you don't know about, that one will still be available for recovery. And that's the problem with all of MS Office software (and other office suites). These office applications create backup copies to recover if (ever) there is a crash.
And you don't ask the shredder to shred them, so they remain on the hard drive, even if you shred correctly the main file. (You can't shred them, because 1° they're necessary 2° you don't know where they are 3° that would be a long job.)

As a conclusion, if you use your shredder for office files such as .doc, .xls and so on, just drop it, it's useless.