How to use US Government standards to ensure your data is secure.

[Darkvision]

Introduction
This is one of those often debated things, how many passes to use with a deletion utility, what encryption standard to use, etc. For this I like to look at current government standards as a base line. Now if it's not super-sensitive data I consider this baseline to be "good enough". However, if you want to be "paranoid" I go with a double of it, "really paranoid" is, well, anything over that. So to me the first thing to address is why even LOOK at US Gov standards as a baseline?


Data Erasure
The U.S Government has spent massive amounts of money and resources on both securing its own data, and breaking into others. Note the end of that sentence: breaking into others' data. This is important, because if the government knows it can get data off say 5 sweeps with a F0 rewrite, it wouldn't use that as its "standard" when clearing its own sensitive information. This also means when you see a standard go up (for instance it used to be 8 rewrites and the disk could be used in non classified pc or even SOLD) they have figured out a flaw/vulnerability with the previous standard and are capable of getting at least some data theoretically from it. Since most here  are looking to secure data are doing it because of the threat of a governments resources being applied to said data it is a good idea to use it as a baseline. One last thing on this, for those not familiar with previous policies degaussing/destruction was NOT always required, this again came about after someone found a way to theoretically still recover data despite even a massive amount of overwrites. how likely/possible this is is UNKNOWN (to the best of my knowledge). That being said, some of what is required is NOT feasible money-wise, like buying your own degausser and destruction apparatus or tech savvy wise, like building your own degausser and disk grinder.


Looking at data erasure we enter a few certain no-nos off the start: putting data that you might want to erase on a SSD is BAD. Do not do this unless you are 100% certain you have a way to get to the hard drive and destroy it. I mean like a way to melt it to slag (blow torch, petroleum jelly, thermite) before it can be taken from your possession. Until a data erasure method is discovered/marketed to wipe, these they are VERY vulnerable. That being said, using a tool like bcwipe on your platter-based HD is useful, so long as you don't do something like a single overwrite of 0s. This is how Bradley Manning got caught. If you see a site telling you "one over write is all you need" don't even bother visiting their site because they have NO clue about securing data. 8 passes using any modern scheme (bcwipe and other programs will have several choices, but I tend to choose DoD cleared methods, as explained above) is VERY secure, but not fullproof. I would say as of now that anything under 12 is far too prone to possible data leaks. This being said, even 200 could potentially leave data, which is why the government has started degaussing and destroying ALL hard drives that contain any confidential or greater information. Since I'm assuming you don't have several grand to throw around for the proper equipment, the next step in securing your data becomes paramount.


Cryptography
Encryption is your friend! But how to choose? Well again this is where looking at why the government chose AES is important. It comes down to at the end of the day, their are not any really successful attacks to use against it. it is VERY secure (currently!). If it's secure enough for TS/SCI information to be encoded in it, I'm guessing your library of furry porn is also going to be safe. Remember the US Government is not worried about hackers reading its mail, its worried about places like China reading its mail, you know China, throwing BILLIONS a year at trying to do this. If the government could crack AES, it would assume china could (or would soon) too. So ignore older standards that are either low key length or known to have attacks against it. That being said, this is about PARANOIA man, and Truecrypt delivers protection in spades. it can encrypt your entire hard disk, encrypt data on the fly and has a number of VERY tough standards to choose from. At the end of the day, if you really want secure data, you need a program like Truecrypt. Now this is one of those amazing head up their asses moments, but the government does not currently encrypt its hard disks, it IS looking to start doing this. However as no current "guidelines" are out we have to try to base this on our own. I choose Truecrypt for a number of reasons: good features, free, 3 top end ways to encrypt your data as well as the ability to cascade those encryptions if you're like INSANELY paranoid. Basically, until quantum PCs hit the market in 20 years your data is secure.




Anyway, end of the day remember that erasure and encryption both have flaws. choosing the right level is up to the end user. I hope this helps people, if I missed something glaring let me know I'll make sure to add it. Below are a whole bunch of links for where im pulling all this from.

Glaring Ommision #1: When it comes to cooking a hard drive for destruction remember that heat destroys magnetic bonds. So even if you're using a 20$ torch from Lowe's it will still "destroy" the drive. For best effects oxy-acetylene is best as it burns hot enough to completly melt the platters. Still for those on low budget, or "good enough" a normal blowtorch will work fine.
P.S. I hope you realize that I do not condone keeping a bunch of thermite around to light your PC on fire to melt your hard drives


P.S.S. If you saw petroleum jelly and started thinking of Vaseline, the "slag" everything approach is not for you, and no I won't tell you how to make your own napalm :p.


Lots Of Links


Data Erasure
DSS Clearing and Sanitization Matrix: This outlines what needs to be done to any HDD/disk.
DoD 5220.22-M: Lots of info on how to handle classified data and its destruction


Bcwipe: The U.S Gov uses this for data erasure, it lists in the program what is "compliant" with federal standards, yes other programs exist that do this with more functionality, but its a good base line program to look at.


Wikipedia article on data remanence: gives some basics on the complications that arise with erasure, which is why the DoD switched to degaussing/erasure/degaussing/destruction for its drives. Again to the best of my knowledge, someone who has done a full say 12-15 passes should be ok, but their are wiggly bits that might be left over, that might be incriminating. NOTE THAT SSDs HAVE MAJOR ISSUES WITH CURRENT ERASURE PROGRAMS!


Cryptography Standards and Stuff
FIPS PUB 140-2: How the U.S government grades encryption standards and a whole lot more.


FIPS PUB 140-2


U.S government replacing DES with AES as its "secure" standard


AES standard


Truecrypt: AES hard disk encryption


edit:kudos to fur for the reformat

[Nerotic7]

I..

[Axon]

I..

You're what? speechless..

[Fur]

Just remember that a cryptographic algorithm is only as secure if the person who implemented it knows what s/he's doing.
For example, a naive programmer may use ECB instead of, say, CBC. That would be bad.

basically until quantum PC's hit the market in 20 years your data is secure.

I don't think quantum computers will have much affect on the security of symmetric ciphers. For more information, try reading this StackOverflow thread. But I'm not a mathematician, so don't just take my word for it.

Also, please fix your grammar and formatting. I found it hard to read that wall of text.

[bluechill]

Just remember that a cryptographic algorithm is only as secure if the person who implemented it knows what s/he's doing.
For example, a naive programmer may use ECB instead of, say, CBC. That would be bad.
I don't think quantum computers will have much affect the security of symmetric cryptography much. For more information, try reading this StackOverflow thread. But I'm not a mathematician, so don't just take my word for it.

Also, please fix your grammar and formatting. I found it hard to read that wall of text.

So actually considering I've studied a fair bit of this in terms of quantum computing and have had teachers who have studied this.  Theoretically it is possible to be a *ton* faster because of quantum states versus finite/definite states (0 and 1).

"It turns out that it's very difficult to contrive a "quantum algorithm" that is faster than any possible classical counterpart."

Very Difficult doesn't mean it's impossible and that's the key.  *IF* you could, then it would be better.  This is exactly like the MD5 collision theory awhile back, people said "Oh there is no way someone could actually do it in practice, it'd be too difficult" and then comes along malware which was probably government sponsored which did exactly that.  So long as it is possible, given an infinite amount of time, it will be done.  (that is also exactly like the thought experiment of what if you given a monkey a type writer and an infinite amount of time, what will he right? He'll eventually write the entire works of Shakespeare and many others.)

[Darkvision]

Just remember that a cryptographic algorithm is only as secure if the person who implemented it knows what s/he's doing.
For example, a naive programmer may use ECB instead of, say, CBC. That would be bad.
I don't think quantum computers will have much affect on the security of symmetric ciphers. For more information, try reading this StackOverflow thread. But I'm not a mathematician, so don't just take my word for it.

Also, please fix your grammar and formatting. I found it hard to read that wall of text.

I think you miss understood the line, the point here is this: give 20 years of advancements on the same "curve" we are on now, traditional pc's will still be no where near cracking AES assuming no major flaws are found. Quantum machines have the ability to render all non quantum cryptographic solutions into a joke. This being said even quantum cryptographic solutions that are out are theoretically secure, but will not be known for sure until tested. Sure considering the ever changing power of computers and our understanding of math and science AES could be rendered obsolete tomorrow. However it is a good encryption standard to use until it is found vulnerable. If you noticed i talked quite a bit about this in the article itself. yes things will change, nothing i have in the article other than complete physical destruction of the disk is full proof. Even then if for instance you ground the disk drive, it is theoretically possible to piece it back together and read the magnetic data, as it is not destroyed. which is why the US Gov insists upon degaussing, running a disk erase utility, degaussing, THEN destroying the disk.


Anyway aside from that i realize im no great shakes as a writer, if you think it needs reformatting, then help me do so. if i like the changes to format i have 0 issue implementing them.

[lucid]

This is one of those often debated things, how many passes to use with a deletion utility, what  encryption standard to use, etc. For this i like to look at current government standards as a base line. Now if its not super sensitive data i consider this baseline to be "good enough". However if you want to be "paranoid" i go with a double of it, "really paranoid" is well anything over that. So to me the first thing to address is why even LOOK at US Gov standards as a baseline?

The Us Government has spent massive amounts of money, and resources on both securing its own data, and breaking into others. Note the end of that sentence:breaking into others data. This is important, because if the Gov knows it can get data off say 5 sweeps with a F0 rewrite, it wouldn't use that as its "standard" when clearing its own sensitive information. This also means when you see a standard go up (for instance it used to be 8 rewrites and the disk could be used in non classified pc or even SOLD) they have figured out a flaw/vulnerability with the previous standard and are capable of getting at least some data theoretically from it. Since most here  are looking to secure data are doing it because of the threat of a governments resources being applied to said data it is a good idea to use it as a baseline. One last thing on this, for those not familiar with previous policies degaussing/destruction was NOT always required, this again came about after someone found a way to theoretically still recover data despite even a massive amount of overwrites. how likely/possible this is is UNKNOWN(to the best of my knowledge). That being said some of what is required is NOT feasible money wise like buying your own degausser and destruction apparatus or tech savvy wise like building your own degausser and disk grinder.

Looking at data erasure we enter a few certain no-no's off the start:putting data that you might want to erase on a Solid State Hard drive is BAD. Do not do this unless you are 100% certain you have a way to get to the hard drive and destroy it. I mean like a way to melt it to slag(blow torch, petroleum jelly, thermite) before it can be taken from your possession. Until a data erasure method is discovered/marketed to wipe these they are VERY vulnerable. That being said using a tool like bcwipe on your platter based HD is useful, so long as you don't do something like a single overwrite of 0's. this is how Bradley Manning got caught. If you see a site telling you "one over write is all you need" don't even bother going their because they have NO clue about securing data. 8 passes using any modern scheme(bcwipe and other programs will have several choices, i tend to choose DoD cleared methods, as explained above) is VERY secure, but not full proof. i would say as of now that anything under 12 is far too prone to possible data leaks. This being said even 200 could potentially leave data, which is why the government has started degaussing and destroying ALL hard drives that contain any confidential or greater information. Since im assuming you don't have several grand to throw around for the proper equipment, the next step in securing your data becomes paramount.

Encryption is your friend! But how to choose? well again this is where looking at why the government chose AES is important. It comes down to at the end of the day, their are not any really successful attacks to use against it. it is VERY secure(currently!). if its secure enough for TS/SCI information to be encoded in it, im guessing your library of furry porn is also going to be safe. Remember the US Government is not worried about hackers reading its mail, its worried about places like China reading its mail, you know China, throwing BILLIONS a year at trying to do this. If the government could crack AES, it would assume china could(or would soon) too. So ignore older standards that are either low key length or known to have attacks against it. That being said this is about PARANOIA man, and truecrypt delivers protection in spades. it can encrypt your entire hard disk, encrypt data on the fly and has a number of VERY tough standards to choose from. At the end of the day if you really want secure data, you need a program like truecrypt. Now this is one of those amazing head up their asses moments, but the government does not currently encrypt its hard disks, it IS looking to start doing this. However as no current "guidelines" are out we have to try to base this on our own. I choose truecrypt for a number of reasons:good features, free, 3 top end ways to encrypt your data as well as the ability to cascade those encryptions if your like INSANELY paranoid. basically until quantum PC's hit the market in 20 years your data is secure.

Anyway end of the day remember that erasure and encryption both have flaws. choosing the right level is up to the end user. I hope this helps some, if i missed something glaring let me know ill make sure to add it. Below are a whole bunch of links for where im pulling all this from.

p.s. I hope you realize that i do not condone keeping a bunch of thermite around to light your PC on fire to melt your hard drives

p.s.s. If you saw petroleum jelly and started thinking of Vaseline, the "slag" everything approach is not for you, and no i wont tell you how to make your own napalm :p.


lots of links:
erasure information:
DSS Clearing and Sanitization Matrix(http://www.oregon.gov/DAS/OP/docs/policy/state/107-009-005_Exhibit_B.pdf) this outlines what needs to be done to any HDD/disk.

DoD 5220.22-M(http://transition.usaid.gov/policy/ads/500/d522022m.pdf)lots of info on how to handle classified data/its destruction

(http://www.jetico.com/products/personal-privacy/bcwipe/) The us Gov uses this for data erasure, it lists in the program what is "compliant" with federal standards, yes other programs exist that do this with more functionality, but its a good base line program to look at.

(http://en.wikipedia.org/wiki/Data_remanence#Complications) gives some basics on the complications that arise with erasure, which is why the DoD switched to degaussing/erasure/degaussing/destruction for its drives. Again to the best of my knowledge, someone who has done a full say 12-15 passes should be ok, but their are wiggly bits that might be left over, that might be incriminating. NOTE THAT SSD'S HAVE MAJOR ISSUES WITH CURRENT ERASURE PROGRAMS!

encryption standards and stuff:
(http://csrc.nist.gov/groups/STM/cmvp/standards.html)FIPS PUB 140-2 how the government grades encryption standards and a whole lot more

(http://csrc.nist.gov/publications/fips/fips140-2/fips1402.pdf) FIPS PUB 140-2(pdf)

(http://www.thefreelibrary.com/Federal+government+finally+adopts+tighter+encryption+standard.-a0110227153) gov replacing DES with AES as its "secure" standard

(http://csrc.nist.gov/publications/fips/fips197/fips-197.pdf) AES standard

(http://www.truecrypt.org/) AES hard disk encryption

(http://www.niap-ccevs.org/pp/pp_swfde_v1.0/) The Us gov search for FDE program

And the phrase of the day is: "That being said.."

Seriously though if someone can't figure out how to make napalm in a very short amount of time then well... you suck.

[Fur]

I think you miss understood the line, the point here is this:

Ah, I get you now.

Still, there is a chance that quantum machines will not break it in our lifetimes, however small it may be. We'll just have to wait and see I guess.

I send the revised thread via PM because other users may find having to scroll through it annoying.

[Nerotic7]

I will be truthful.
I didn't read a single thing on this, but kudos to you.
I can see that you have put time and effort into this.

[lucid]

I am interested why it's a bad idea to use an SSD when up until now I was lead to believe that they are what you should use to securely delete files.

[Darkvision]

I am interested why it's a bad idea to use an SSD when up until now I was lead to believe that they are what you should use to securely delete files.

It has to do with how the architecture of the SSD differs from that of a platter array as well as how they deal with information/writing. Its not that it CANT be secure, but that problems have been found with both full erasure, as well as single file deletion.  Anyway if you look at the wiki link it can give you a brief overview, or check its source. here are some additional links


http://forum.crucial.com/t5/Solid-State-Drives-SSD-Knowledge/SSDs-and-Secure-Erase/ta-p/112580 --cruicial on SSD's and erasure


two tech articles on it:
http://www.tomshardware.com/news/solid-state-flash-translation-layer-NAND-FAST-11-Sanitization,12252.html
http://nakedsecurity.sophos.com/2011/02/20/ssds-prove-difficult-to-securely-erase/


End of the day, its just not a good idea to use SSD's for "sensitive" data storage right now. I am sure this will change when new methods are developed for secure erasure. They do have the potential of being safer than a platter based system, it is just that currently they are less secure.


edit: figured i would also point out that built in disk erasure utilities on the drive itself CAN be safe according to some studies, but it requires you to test each and every drive type to see if it is secure/if data is left over. As i dont know the high end of this field(data recovery), and im not sure anyone here does, it would seem safer to me to just not use SSD's even if you think the built in program does it's job correctly.

[trexd___]

well if your really keen on security use one time pads all the time so even if aliens with magical super computers came they still couldn't crack it.   

[vezzy]

well if your really keen on security use one time pads all the time so even if aliens with magical super computers came they still couldn't crack it. 

What is snake oil?

[trexd___]

What is snake oil?

"One-time pads are a popular cryptographic method to invoke in advertising, because it is well known that one-time pads, when implemented correctly, are genuinely unbreakable."
-[size=78%]http://en.wikipedia.org/wiki/Snake_oil_(cryptography)[/size]

[vezzy]

Good job quoting Wikipedia!