How to Permanently Delete Data from Your Hard Drive
According to the New York Times, a basic privacy measure that is often overlooked is the proper destruction of data on hard drives. An ongoing study by British Telecom says that most people don't realize that deleting a file doesn't actually remove the data from a computer.
In fact, the BT research found that only 33 percent of second hand hard drives had been completely wiped clean. To ensure your drive doesn't contain any personal data before you give it away or sell it, you need to reformat the hard drive or use digital shredding software if you want to completely eliminate all traces of data. In this post, we'll show you how.
When it comes to data stored on your computer, deleting files doesn't actually remove the data. File information is kept in a directory so that the operating system can find it. When you delete a file, all you are doing is removing it from the directory and flagging that part of the drive as being available for new data. Until that region is overwritten, the old data can be retrieved, in fact that's how you can recover lost data. It's also the way most file recovery programs work - they look for data on your hard drive that shouldn't be there according to the directory and restore it.
The only way to completely remove the data is to overwrite the contents of the hard drive. You can do this by formatting the drive, or using data wiping software that fills your hard drive with random data.
Two Main Methods of Overwriting Data
The Gutmann Method
Based on Peter Gutmann's paper "Secure Deletion of Data from Magnetic and Solid-State Memory", this method provides the best security. Data is overwritten 35 times with carefully selected patterns, which makes it unrecoverable. Unfortunately, it also makes it time consuming.
US DoD 5220-22.M
Based on the United States Department of Defense recommendation 5220-22.M, this method overwrites the data seven times. While less secure than the Gutmann method, it is faster.
Mac Users: Data Wiping Built In
Macs come with data wiping tools built into their systems. To securely delete a folder or file, all you need to do is move it to the trash can and from the Finder menu select 'Secure Empty Trash'.
Alternatively, the Disk Utility program 'Erase Free Space' scans your hard drive for unused space and securely deletes it to military (7 passses) or Guttman (35 passes) standards. You can find it in the Utilities folder.
Windows: Data Wiping by Selected File/Folder
Eraser is a free, open source program that works with Windows 95, 98, ME, NT, 2000, XP, Vista, Windows 2003 Server and even DOS. It uses carefully selected patterns to overwrite your hard drive several times and lets you select single files, entire folders or the entire drive to be wiped clean.
How to Use Eraser
Download and install the application. Once running, you'll be presented with a simple box that allows you to either schedule an erase, or do it there and then (on demand).
There are three ways to enter data into the list if you are doing it on demand:
1. Select files and folders in Explorer and drag and drop them to the list
2. Copy them to the clipboard and then paste them to the list
3. Use the New Task command in the File menu.
Note: If you use the New Task command, a window will appear allowing you to select unused space on a drive, a folder or a file to be erased.
After you have added the data you wish to erase to the list, you need to select the method of removal.
- The Gutmann Method (Default)
- The US DoD 5220-22.M Method
- The Pseudorandom Data Method
Using the pseudorandom data method, all passes will be random data, which is highly incompressible. This is the only method that should be used when erasing unused space or data on a compressed drive. The number of passes is user selectable from one to 65535.
Once you've selected the files/folders/drives that you want deleted, and the method of deletion, you can run Eraser. It will ask you for confirmation before it starts erasing so make sure you are certain you want to continue as this is your last chance to prevent data from being accidentally erased.
Data Wiping of Complete Drive
Another free open source program that runs on Windows virtually any machine, Darik's Boot and Nuke (DBAN) wipes the entire contents of every hard drive it finds on your computer when you run it, so be very careful.
Darik's Boot and Nuke ("DBAN") is a self-contained boot disk that securely wipes the hard disks of most computers. DBAN will automatically and completely delete the contents of any hard disk that it can detect, which makes it an appropriate utility for bulk or emergency data destruction.
DBAN is simple enough to use. Once you've downloaded the program, write it to a CD, DVD, or thumb drive, boot from that and enter 'autonuke' at the prompt.
The New York Times has an in depth article about DBAN that includes an interview with the author, Darik Horn if you are interested in further reading.
The Ultimate Permanent Erase
While these methods of data wiping can be useful for most of us; if you're uncertain about using them, there is one other option.
- Remove the hard disk from the computer
- Unscrew the casing, exposing the disks
- Smash them to smithereens
Your data now remains private. :)
Update: As pointed out by Stu in the comments, simply formatting the contents of the hard drive is insufficient if you want to delete the data; the drive will also need to be overwritten to ensure data deletion. Additionally, a 'quick-format' will not suffice, it must be a full format.
Share
Comments
Subscribe to comments for this post OR Subscribe to comments for all ReadWriteWeb posts
Hi,
Your article mentions formatting a drive as a way to erase data stored on it. I'd strongly recommend qualifying that comment; if you only quick-format a drive, then your data is still there to be retrieved using suitable tools.
You might also find the Great Zero Challenge worth a mention. This Challenge aims to determine whether or not data can be recovered after a drive has been wiped just once with zeros. The folks behind the Challenge assert that it isn't possible to recover data after this procedure, and are challenging professional data recovery companies to prove them wrong. After 10 months, the Challenge still hasn't been answered.
http://16systems.com/zero/index.html
Best regards,
Stu
The one thing worse than wrong information, is wrong information posing as fact. Despite the good intentions of the author, there are a number of errors in this article that would lead someone seeking advise to actually put their data at risk.
1/ FORMATTING a drive does NOT destroy the data.
2/ DoD 5220 is outdated guidance. In the latest revision of the DoD 5220 guidance, resposnsibility for guidance for data sanitization is referred to the National Institute for standards and technology, specifically their special publication 800-88. In NIST Special report 800-88, up to date guidance is provided for the sanitization of all types of storage.
3/ DoD 5220 is a 3 pass process, not 7 as stated in the article. The first pass being all '1's, the second all '0's and the third pass bieng random data.
4/ recommending that the public opens the Head Disk Assembly and uses physical force to smash the platter is not safe nor necessarily best practice. The media and HDA are brittle and using physical force can cause fragmenting that can result in bodily harm from the resulting shrapnel. Additionally, depending on what type of data you are trying to destroy, mere smashing the media is not a recognized means to decommission higher levels of classified information.
When handling any confidential data that requires absolute destruction, Government policy requires absolute destruction. Absolute destruction is defined as the destruction of the media beyond recovery by any means. If using physical destruction methods, this would mean that the media is reduced to a particle of a diameter no greater that 1/250th of an inch, the diameter smaller than would be required to accommodate a single 512 kb data block. Consequently most contract physical destruction providers fail to meet this screening size.
Another concern is defining the best practice for the sanitization of drives with different interface types. Software based utilities are incapable, that is correct, incapable of purging all data from ATA type devices. This means that no matter whose utility you buy, it will NOT eliminate all writable data from the media surface.
Not only are software based utilities slow (approximately 8-12 hours for a 100 Gig drive), but at the end of the process, recoverable data will exist. Read vendor claims carefully. This is also why the NIST guidance is never stated, it just does not cast a good light on the software.
The reason software is not fully effective on ATA devices is due to the design of the ATA (IDE, SATA, PATA, etc.) drives. These devices are engineered to provide low cost high capacity storage, and have integrated controllers optimized to delivering these values. features such as Protected service areas, including the Host Protected Area (HPA) and Device Control Overlay (DCO), as well as G-List (bad block list -growth) cause storage regions on the media surface to be protected from access by software based read/ write commands that are used by software based utilities to accomplish their goal.
SCSI type devices use an external controller model where the controller can be instructed to read /write to specific sectors without restriction. In this case, software based utilities are an effective, yet slow, means sanitize the media, as there are no access restrictions imposed by the controller or the host.
Recogizing the limitations posed by the ATA spec, and as a means to establish a standards based protocol for the sanitization of hard drive storage, 6 drive manufacturers, and the University of California San Diego's Center for Magnetic Recording research had developed a technology called Secure Erase/ Secure Initialize. This technology is actually embedded in every standards based IDE, ATA,SATA hard drive produced since 2001, as part of the ATA spec. Secure Erase was designed to be an efficient and easy to use data sanitization protocol that provides PURGE level sanitization (the same level of data destruction as an effective degausser). As a highly effective and efficient process, Secure Erase is capable of purging a 100 Gig drive in less than 30 minutes. The resulting device being both reusable and void of any recoverable data (by any means).
Secure Erase, although being highly effective is protected from execution by hardware and BIOS level protection as if exploited by virus or malware would be devastating. When used in the enterprise or government, the best means to launch Secure Erase is by using appliances engineered to run Secure Erase.
Referencing the guidance from the UCSD CMRR, one such vendor who is producing such an appliance is Ensconce Data Technology in Portsmouth New Hampshire (USA).
A lot has been written about data destruction, yet, short of knowing where to find current and clear guidance, a lot of misconceptions exist. Perpetuating information that is not factual or based on accurate data adds to this confusion.
I have conducted a lot of research into data destruction practices and the risks posed by the various methods and practices prescribed. I will gladly provide reference and substantiation to any claim I have stated. I can be reached at ryk@converge-net.com for more information.
So! What we gonna do by now?
Could standart formatting help us? Is it okay?
Can someone explain to me the need for multiple passes of writing different patterns.
Say a portion of my disk has data on it. e.g "11000011110101".
I overwrite it with zeros once, so it is now, "00000000000000".
Now tell me, how an earth can someone get hold of my original data??????
DBAN program erases data completely so that data cannot be recoverable,its good to use DBAN to protect your privacy so that no data won't be recovered.
Apparently (no source offered) if you overwrite data with all 1s or 0s the overwritten data doesn't completely overwrite the previous write path and the edges of previous data is/are visible. NSA use edge reading to recover data.
Um, DBAN doesn't run on Windows - it's a "self-contained boot disk", as it says in the quote from the about page (right below where you said "runs on Windows").
It'll run on any PC, whatever the OS (or even if there is none).
Windows 2000 and XP Pro users can overwrite deleted data using cipher.exe from the command line. This utility makes a three-pass overwrite of all free space on the drive - first with zeros, then with ones, then with random numbers. It DOES NOT touch anything that hasn't been "marked for deletion" by Windows. :
http://support.microsoft.com/kb/315672
http://www.windowsecurity.com/articles/Using-cipherexe.html?printversion
If you have XP Home, find someone with XP Pro and copy the cipher.exe file from the windows/system32 folder to a thumb drive. Then, copy the file to the same directory on your XP Home box. Works great!
Stu, thank you and you’re absolutely right.
I had the quick format issue at the back of my mind while putting this together, and for whatever reason forgot to include it. I should have included it; I should have also included the fact that the reformatted drive still needs to be overwritten. Sorry, and thank you; I've now updated the post.
However, I LOVE what I’ve read about the Great Zero Challenge. I will have to take a more detailed look at that. Thank you so much for pointing it out.
Ryk, thanks for spending the time writing this up.
Just to be clear, the article was written from the point of view of an individual giving away or selling their hard drive, not enterprise or government, who would certainly need to do much more than I have offered here (although your summary would possibly be a great start for them).
As you would have to agree, recovering deleted data isn’t an automatic, simple or even timely process. You need the right software; a knowledge of how to use it, as well as the time to sort through the jungle of data to get to the ‘juicy’ stuff.
You are right about formatting, and again, as I did with Stu, I apologize as I really should have qualified that better.
Regarding US DoD 5220-22M, as you rightly point out, the standards are outdated, however, the methods described are still used by many; I believe both DBAN and Eraser use the method as part of their overall data deletion process.
In terms of number of passes, and from my limited knowledge, I understood that the US DoD short wipe uses three passes, but the US DoD US 5220-22M standard wipe has seven passes. I took this information from the California Polytechnic State University (PDF here).
In terms of disassembling the drive, physical force is probably not the best solution, however, my point is that to be truly safe, there is only one way, and that is to completely destroy the drive.
Again, thank you for providing this information, I knew various corporations and government departments used hard drive shredding service to completely annihilate the data; I just didn’t realize to what extent.
Solaiman – I wish I could answer you, alas I cannot – perhaps Gaz’s answer helps?
Martin – you’re right. It does run on virtually any system. It was careless of me; I apologize and will fix it.
Mechmike – brilliant. Thanks for letting us know!
The Unix Direct data (dd) command, anyone?
Response on Track Edge Phenomena:
The track edge phenomena is not a valid concern. This is an outdated concept that died many generations ago. The concern of hard drive manufacturers is data density, and leaving margins is a waste of usable space. What once had been a significant sized margin is now nothing more than microns containing nothing more than magnetic noise.
Please don't take my word for this. Reference comments from the Center for Magnetic recording research at the USCD.
Multipass process:
The concept behind multi-pass overwrite processes stems from the concept of elimination of magnetic artifacts. By writing all zeros, all bits are set to null, the following precess is set to a binary 1. Ultimately after these 2 passes all bits should be uniform. The final pass assures that should any magnetic artifact remain, that it will be obfuscated by random data.
It has been stated that a single pass process is satisfactory in sanitizing data effectively. The issue comes down to making sure that all data sectors are processed. Hardware and controller limitations can pose impedance in making sure that all data storage regions are addressed. Products such as DBAN and others will in many cases address most users needs. However, when used to deliver compliance solutions, or for the protection of enterprise or government data,concerns come down to:
- the certification of the application each time it is launched.
- validation of the software image to assure against intentional tampering or software corruption.
- reliable auditing features that assure a tamper proof method of tracking processed assets.
- A scalable solution that can be deployed on-site that assures absolute care, custody and control using an efficient process.
Encryption:
Encryption is a great means to protect active data, but is not a recognized means to sanitize end of life data. When handling classified information where absolute destruction is required, the process must render the legacy data unrecoverable by any means. This means that technologies such as encryption which, regardless of key complexity, can reconstitute the original data.
Formatting:
re initializes the file allocation data and simply obfuscates the file location. Yet the data still resides on the disk. Recovering this data is relatively simple and can be done with off-the-shelf recovery software or by a professional lab. Results are very good, unless the sectors are overwritten. In this case, data in non-reallocated sectors can be recovered.
The one thing worse than wrong information, is wrong information posing as fact. Despite the good intentions of the author, there are a number of errors in this article that would lead someone seeking advise to actually put their data at risk.
1/ FORMATTING a drive does NOT destroy the data.
2/ DoD 5220 is outdated guidance. In the latest revision of the DoD 5220 guidance, resposnsibility for guidance for data sanitization is referred to the National Institute for standards and technology, specifically their special publication 800-88. In NIST Special report 800-88, up to date guidance is provided for the sanitization of all types of storage.
3/ DoD 5220 is a 3 pass process, not 7 as stated in the article. The first pass being all '1's, the second all '0's and the third pass bieng random data.
4/ recommending that the public opens the Head Disk Assembly and uses physical force to smash the platter is not safe nor necessarily best practice. The media and HDA are brittle and using physical force can cause fragmenting that can result in bodily harm from the resulting shrapnel. Additionally, depending on what type of data you are trying to destroy, mere smashing the media is not a recognized means to decommission higher levels of classified information.
When handling any confidential data that requires absolute destruction, Government policy requires absolute destruction. Absolute destruction is defined as the destruction of the media beyond recovery by any means. If using physical destruction methods, this would mean that the media is reduced to a particle of a diameter no greater that 1/250th of an inch, the diameter smaller than would be required to accommodate a single 512 kb data block. Consequently most contract physical destruction providers fail to meet this screening size.
Another concern is defining the best practice for the sanitization of drives with different interface types. Software based utilities are incapable, that is correct, incapable of purging all data from ATA type devices. This means that no matter whose utility you buy, it will NOT eliminate all writable data from the media surface.
Not only are software based utilities slow (approximately 8-12 hours for a 100 Gig drive), but at the end of the process, recoverable data will exist. Read vendor claims carefully. This is also why the NIST guidance is never stated, it just does not cast a good light on the software.
The reason software is not fully effective on ATA devices is due to the design of the ATA (IDE, SATA, PATA, etc.) drives. These devices are engineered to provide low cost high capacity storage, and have integrated controllers optimized to delivering these values. features such as Protected service areas, including the Host Protected Area (HPA) and Device Control Overlay (DCO), as well as G-List (bad block list -growth) cause storage regions on the media surface to be protected from access by software based read/ write commands that are used by software based utilities to accomplish their goal.
SCSI type devices use an external controller model where the controller can be instructed to read /write to specific sectors without restriction. In this case, software based utilities are an effective, yet slow, means sanitize the media, as there are no access restrictions imposed by the controller or the host.
Recogizing the limitations posed by the ATA spec, and as a means to establish a standards based protocol for the sanitization of hard drive storage, 6 drive manufacturers, and the University of California San Diego's Center for Magnetic Recording research had developed a technology called Secure Erase/ Secure Initialize. This technology is actually embedded in every standards based IDE, ATA,SATA hard drive produced since 2001, as part of the ATA spec. Secure Erase was designed to be an efficient and easy to use data sanitization protocol that provides PURGE level sanitization (the same level of data destruction as an effective degausser). As a highly effective and efficient process, Secure Erase is capable of purging a 100 Gig drive in less than 30 minutes. The resulting device being both reusable and void of any recoverable data (by any means).
Secure Erase, although being highly effective is protected from execution by hardware and BIOS level protection as if exploited by virus or malware would be devastating. When used in the enterprise or government, the best means to launch Secure Erase is by using appliances engineered to run Secure Erase.
Referencing the guidance from the UCSD CMRR, one such vendor who is producing such an appliance is Ensconce Data Technology in Portsmouth New Hampshire (USA).
A lot has been written about data destruction, yet, short of knowing where to find current and clear guidance, a lot of misconceptions exist. Perpetuating information that is not factual or based on accurate data adds to this confusion.
I have conducted a lot of research into data destruction practices and the risks posed by the various methods and practices prescribed. I will gladly provide reference and substantiation to any claim I have stated. I can be reached at ryk@converge-net.com for more information.
I've always wondered about data deletion on a hard drive that has developed some bad sectors. Is it possible that the failure of the physical media could leave it in a state where old data could not be completely overwritten using software?
I like The Ultimate Permanent Erase. Very efficient, and quick.
The Ultimate Permanent Erase does take care of the issue either, I have personally seen a floppy disk be burnt and data still recovered. I have seen and watched a recovery company take a hard drive that I know has data on it (it was mine, I formatted it 3 times first) open it, break the disk into p
If I'm going to sell or donate an old computer, I don't bother to rewrite my data anymore. I simply purchase a new drive.
Back in the day, this was excessive, but today, you can buy a decent HD for around 40 bucks (significantly less if you use Ebay). I just build that price into the computer that I'm selling, or just absorb it as peace-of-mind tax.
I can honestly say that with this method, my data is guaranteed 100% secure. I still own those old drives - some have been totally destroyed, others converted to external HDs and seeing new usage. Nevertheless, I KNOW where my old data is and am still in complete control over it.
Bad Sectors:
Sectors flagged as bad at the time of manufacturing are referred to as Permanent List (P-List) bad sectors and in the case of ATA (IDE, SATA, and PATA) devices, these sectors are not allocated to the LBA (Logicial Block Address Table). This means that any sector discovered as bad at the time of manufacturing has no logical address and no data can be written to these blocks.
Sectors that are identified as bad during the operation of the device are added to the Growth list (G-List)table. These are teh sectors that you should be concerned with as they are the sectors where a failed data write occurred, and may contain a partial block write. Once a sector is flagged in the G-List the original data that was to be written to this sector or track is redirected to a region dedicated for accommodating slip track data. The original block is then flagged as unusable by the drive controller.
The G-List can exist either in controller flash, or on the media surface, and reloaded on power up to the controller flash. In either case, data recovery tools or labs can recover information in these sectors where the state of the data in the block is of sufficient quality to reconstitute some or all the information.
well, everybody seems to bother with hard drives, while new items more and more often have a Flash (SSD) inside, and there rules of engagement are different. just before you've decided to get rid of your EEE pc -- bother yourself with searching methods to do wiping ;)
Quite useful information thanks for the Post
This is absolutely stupendous post. I'm among the 66% ppl who think data is deleted when I delete a file. Now downloaded erase and trying it out.
BTW, love the last tip abt smithereens :)
I recently came across your blog and have been reading along. I thought I would leave my first comment. I don't know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.
kaylee
http://www.thinkpadonline.info
Paranoid - Good point. I really don't know - sorry. But it would be interesting to find out
Chris - that is just scary
Bridge - Yeah, it is like that. We've got old hard drives (and old hardware) floating around this house. Which makes me think maybe I should be getting rid of it. :)
Ryk - Thank you for clearing it up.
ShriNagesh - Thanks ;) It is scary - I know some of my friends and family wouldn't even think about it - wouldn't realize. The fortunate thing with you though is that you're learning about it; others - sadly yes my friends - don't think it is worth worrying about.
Kaylee - well done on your first comment! here's a tip - if you put your URL into the filed below your name - your name will automatically be linked to your site. :) Thanks for stopping by
Kaylee - that should be 'field' not 'filed' :)
Data entry is a fast growing industry. The world of business is dynamic, fast paced, and in constant flux. Is provide by Real data assistance.com
Data entry service providers