How does a file get “corrupted” and how can I prevent it?
In: Technology
A corrupted file is one where data got wrongly copied/saved, making it unreadable. For example the header that tells the computer how to read the following data has a small error so the remainder of the file seems like gibberish because it’s interpreted wrongly.
Aside from not pulling the power cord while saving files there is not a lot you can do. Errors happen sometimes, especially when you download stuff through a shaky connection or stuff like that.
The easiest way to deal with it is to simply keep backups of everything important. If something breaks you can simply get the original back.
Preventing file corruption starts with proper device care. Don’t allow your devices to get too hot or too cold, as either could damage their physical hardware. Also, if part of your daily routine involves a lot of downloading or file transfers, consider installing antivirus software and some form of anti-malware to protect your data. This might seem fairly common sense, but it’s especially important when considering the integrity of your files and data.
A file can be corrupted in any number of ways. The simplest way is that the contents of the file is wrong. Maybe the file got truncated, so it’s shorter than it should be. Maybe the values in the file are wrong, thus rendering the contents nonsense to the program that uses that file.
Worse, the filesystem can be corrupted. Your drive is just a device with a means of setting and reading bits. A filesystem is software and data that manages your files, tracking file name, location, size, timestamps, etc. THAT can get corrupted, rendering the file effectively unrecoverable; even if the file contents are still on the drive and just fine, it doesn’t matter if the filesystem loses track of the pieces.
Corruption can happen by the program that uses the file, if the program has a bug. Corruption can happen by other programs doing something wrong, or perhaps a program misguided by a naive user. Corruption can happen due to hardware failures. Corruption can happen due to interference, RF noise, solar flares, etc.
How can you prevent it? You can’t. You can reduce the chances of corruption. You can possibly even recover from some kinds of corruption with the right preparation. Prevention is going to be your first defense. Backup your data. Properly. You can always buy a new computer, new software. You can’t buy back your only unique copy of un-reproducible, irreplaceable data.
The second thing you can do is use technologies that help reduce instances of corruption. EEC memory is one example (though iCore and I believe Ryzen both don’t use the EEC features even though some motherboards can use the memory). RAID-6 is another example. This doesn’t just write bits to your drive, it encodes it, so that if there is corruption due to hardware or solar flares, they can be detected and possibly even recovered. RAID-6 uses multiple drives, can can even withstand 2 drive failures at the same time. (RAID does not constitute a backup solution. No, not even RAID-1.) If you’re really curious about this, I’ll just name drop Ubuntu and BRTFS.
The third thing you can do is – don’t do anything stupid. If you’re going to run `rm -rf *`, make sure you’re not in the root directory. Test what you’re going to do, if it’s irreversible, before you do it. Read your manuals. Know what you’re doing.
Digital data is converted into a series of “on’s” and “offs” or 1s and 0s via different very small phenomena. These small bits of storage can be fairly sensitive to outside changes, and sometimes can be flipped to the opposite state almost randomly, or through interactions with very weak particles. The end result can change stored value of a character or number in a file to something else including many nonsensical terms. When the file in turn is read from storage sometimes those errors prevent the file being correctly understood to be opened. For some text files you can often open them into simpler text editors to see all the characters and remove the offending element. Other times you can’t. There are many systems to help automatically correct these errors but they more mitigation strategies than anything else.
For something that has been stored long enough the accumulative collection of these rare and random effects can really make recovery more difficult if not outright impossible. The best ways to store things for long term is to make multiple “pristine” copies and store them separately. When it comes time to read them you have a system to check all of them side by side. Wherever one of them is different from all the others, that’s an error that can be ignored.
As you probably know, digital files are stored as a series of bits (ones or zeros). To use a very simplified version of how files are usually structured, an image file might start off with a pattern of ones and zeros that says:
“This is an image file. This label is 500 bits long and the actual image starts at the 501th bit. The actual image is 3840000 bits long. Starting on the 501th bit, there are 8 bits telling you how red the first pixel is, then 8 bits telling you how green the first pixel is, then 8 bits telling you how blue the first pixel is, then repeat for the second pixel. The image has 400×400 pixels.”
When a program tries to open this file, it might notice “hey, the label at the start of the file says the file has a 500 bit label followed by 3840000 bits of image, so the file should be 3840500 bits long, but the file is actually 3840476 bits long. I don’t know how to interpret this. Error.”
That would be an example of file corruption.
A file’s label might also say “by the way, if you add up all the ones in this file, the total should be 365903.” If some bits in the file have been accidentally changed (e.g. due to the computer’s memory getting hit by cosmic rays, which is a real thing), then that will usually change the total number of ones, so it won’t match the total recorded in the file label, and the computer will know the file has been corrupted.
Other common ways you get a corrupted file are when the program that was creating the file got unexpectedly shut down and left the file half-written, or when the computer mistakenly thinks that the file is another type of file, sort of like if someone writes down their phone number for you but you mistakenly think they’re writing down their address: if you were a computer, you’d look at the piece of paper, see “123-555-7809” and say “this is not a valid address, must be corrupted.” The number is in fact a perfectly valid phone number, but because you have been told it’s an address, you treat it as corrupted.
Latest Answers