why does it take longer to transfer hundreds of files totalling 4 gigabytes than transferring one big 4 gigabyte file?
In: Technology
Overhead
Each individual file is not only the data it contains but also it’s corresponding entries in the NTFS file table.
If you think of a hard drive like a library, then the files are books, and the FAT (File Table) is the index that tells you where to find these books.
Every time you write a file you have to update the index as well, and that adds overhead. So imagine being in a library and having to put a couple hundred books back on the appropriate shelves. You have run back and forth between the shelves and the index, putting the books away and then updating the index. It’s not very efficient. Vs putting a single set of encyclopedias back on the shelf in order.
This is worse on spinning hard drives because the hammer has to move around a lot to access the index, then write the file. Each move of the hammer is time spent not writing data to disk, that’s latency and it adds up.
On top of that the built in Windows File transfer utility doesn’t handle large file transfers well, so the amount of time it takes to move data increases on a curve instead of a straight line.
When I’m migrating a lot of data in the server world I use a tool call Teracopy (which is free for home users) that replaces the built-in Windows File utility for this very reason.
Latest Answers