with content being created on a daily basis and posted to social media or any other outlet, how is it all stored and managed? Will we ever run out of ‘space’?

679 views

with content being created on a daily basis and posted to social media or any other outlet, how is it all stored and managed? Will we ever run out of ‘space’?

In: Technology

TLDR: Yes, but then they’ll just add more.

Speaking as a Storage Administrator working in the IT industry, if you give users space they will find a way to fill it.

The data stored by web-based apps of this scale are typically in the form of databases and file stores located on dedicated storage appliances rather than individual servers. Think of this as a server rack full of hard drives controlled by a device that allows multiple servers to talk to it simultaneously. We call this a SAN (Storage Area Network)

These devices have massive amounts of space, often have built-in compression and de-duplication (replacing multiple copies of the same file with a single instance of that file) and are able to be expanded dynamically. They also include different Tiers of storage, files that are accessed frequently are stored on fast but expensive SSDs (Solid State Drives) and less used archived data is storage on slower but significantly cheaper spinning disk hard drives.

Basically you can buy another tray full of hard drives and click it in as you need it.

All content on the internet is stored on computers not unlike the one you have at home, but with a lot more hard disk space and usually more processing power as well. These are your web servers. Companies maintain huge rooms full of servers and sell access to that storage to other companies. Since engineers are coming up with more efficient storage all the time, you can have a huge amount of data storage in a very small physical space. It is ridiculously cheap to store data nowadays. So no, the chance of running out of space is relatively low. A bigger concern is the durability of the data. Hard drives can degrade over time and are susceptible to data loss from physical disturbances (say, an earthquake or simply dropping it). That means a good data center is built with redundancy in mind, maintaining multiple copies of the data and even copying it to analog storage medium like tape.

Edit: Software also keeps getting better at storing more information with less “data units”, i.e. bytes thanks to clever compression algorithms. So the space you need to store a high-resolution image today is a lot less than what you would have needed 10 years ago.