How does YouTube, Google, etc. seem to have ‘unlimited’ storage for their users?
In: Technology
They have a lot of data centers, which are basically warehouses with a lot of hard drives and teams of people who can plug in more hard drives faster than their users are filling them up.
Large scales are often hard to comprehend, that’s just part of being human. Let’s work our way up.
You can typically squeeze a PB into a height of 4U (4 rows in a server rack).
[This is a server rack](https://www.racksolutions.com/news/app/uploads/bg.jpg). They can come in sizes of up to 90U! Many of them are put next to each other for easier management and then that’s called a data center.
[That’s one of Google’s data centers](https://wp.technologyreview.com/wp-content/uploads/2018/08/googlecbf009-11.jpg).
Google owns and maintains many many data centers.
Also do note that they also do not offer actual unlimited storage. Everything [can and will be taken away at their convenience](https://www.cnet.com/tech/services-and-software/google-photos-unlimited-free-storage-is-gone-heres-how-to-get-more-space/).
they don’t exactly have ‘unlimited’ storage, free Google account gives you only 15GB all across GMail, Photos, and GDrive (Photos used to be separate, but now they combined them)
YouTube is also another case, they probably have around several hundreds Petabytes of videos, possibly more, and it’s growing *each second* as people uploads their videos.
How they do this?
[They have data centers all across the world](https://en.wikipedia.org/wiki/Google_data_centers)
each one is growing and can probably store a crazy amount of data, maybe several Exabytes or so (1 Exabyte = ~1000 Petabyte = ~1,000,000 Terabyte)
They also compress those data and have [their own filesystem](https://cloud.google.com/blog/products/storage-data-transfer/a-peek-behind-colossus-googles-file-system)
Don’t underestimate what money can buy lol, especially if you have billions like Google.
They actually have teams whose job it is to do capacity management. In simple terms, they figure out with very high confidence the most storage they’ll need in the future and make sure they have what they need in place to support that.
While an individuals storage needs are hard to predict, when you’re talking these scales, things tend to average out. Person A uses a little more than you expect, person B uses a little less etc and they have historical data to know if certain time periods vary. This gives them a good model to know what they need.
They definitely do not. The current status quo where users have come to expect that storage is free is inherited from an era when various growing companies – such as Google themselves, but also companies like Dropbox – were fighting to get users and were giving away storage for free. During the first years, GMail had a free quota that tended to double every few years – now it has stayed frozen for the last 10 years.
Now, it is a trend that many companies would like to see reversed – and they probably will – as it is becoming more and more of a problem. The price of storage was falling very fast during the early 2000s, but now has stabilized – and there were even a few bumps as storage transitioned from hard drives to solid state.
The era of free storage is coming to an end. YouTube still resists, but YouTube is probably the company that has the best monetization of the content they have to store for free.
Your computer has a hard drive. Maybe two.
They have hundreds of computers and *thousands* of hard drives.
There is no clever trick here. They literally went out and bought *that much* storage. That’s it.
All the providers have a couple “tricks” they do to cut costs and restrict abuse.
First of all, if you are uploading a file that already exists on the system (detected by at least two strong hashing algorithms); they just keep a single copy of it and then use a pointer to it.
Second, if you try and upload hundreds of gigabytes of stuff they will very quickly throttle you in the hope that will just cancel the upload. They also won’t keep partial files.
Well, with YouTube the videos are the product and make them money, so they’re probably willing to host a lot.
Gmail does not have unlimited storage, it’s capped at 15 GB across Gmail, Google Drive, and Google Photos. Text just doesn’t take up very much space so you can save a lot of emails in 15 GB, photos take up space faster. If you want more than 15 GB you can set up an Google One account. 100 GB is $19.99 a year.
Imagine your mom says she will buy you a set of Lego blocks anytime you want as long as you have enough room on your shelves. And you figured out that shelves are a lot cheaper to buy than expensive Lego sets you keep buying and putting up shelves, and she keeps giving you more Lego sets.
You have an incentive to make sure you have more storage space than you need so you can keep getting more stuff to put in them.
Same with Google. They make money off of the stuff people put in their storage – either by virtue of people paying for that storage or by them profiting from what is hosted there such as the money they make from ads on YouTube videos. And they make more money off of what you put there than it costs them to host it. So they have every incentive to continue to increase their storage to make sure they never run out so people can keep putting more stuff in them. They are constantly building new shelves (data centers) and extending to make bigger shelves out of the ones they have (adding more capacity to their existing data centers, upgrading hard dives to bigger storage, etc).
while they do have large data centers, they are trying to cut down on how much data they actually keep stored, for example, by deleting inactive accounts and such
Latest Answers