Recently I learned that some file systems support compression for the entire drive/partition. I was curious to find out that my smartphone has compress\_algorithm=lz4 attribute in f2fs for the /data partition. So my question is how does it work? Am I really able to put more than 110GB of data on such a 110GB partition due to compression? What do I see in the space usage stats? How will it handle case if I need to read some big file while there is no free space to uncompress it?
In: 3
Filesystem compression usually occurs at a block level
Files stored on a filesystem are made of individual blocks. Each block might be say 64k in size
The filesystem will detect any duplication within the block, and only store that block once. Each time that block appears again it is instead replaced by a pointer.
Imagine it this way
Take any given book:
Go through the book and find the most common word, say ‘the’
Replace all instances of ‘the’ with a symbol like ‘@’
Then at the start of the book put an index that reads @ = ‘the’
If there’s 300 examples of ‘the’ in the book you just saved 600 characters (60% of a Kilobyte) of data without losing any information.
Repeat this over and over again for repeated words and you can compress the book down by A LOT
The same goes for the filesystem. You don’t need to store the same repeated information over and over again, even parts of a file.
You don’t need to decompress the file either, you just pull the blocks required on demand into RAM
Yes, file system compression allows you to store more data on a partition by reducing the file size
However, the space usage stats may not accurately reflect the compressed file sizes
In case you need to read a big file without free space, it will be uncompressed on-the-fly for access.
Latest Answers