How does YouTube handle millions of video notifications at once, while Discord can barely send out a couple thousand @everyone?
In: 13
YouTube is owned by one of the largest tech companies on the planet. Discord was started 7 years ago and, while popular, is quite small in comparison.
Google has 23 data ~~servers~~ centers worldwide, likely with *billions* of dollars of equipment in each one, Discord is worth around $15 billion total. They might have worldwide servers through cloud services, but they can’t compete with what Google has access to.
One thing I would add, is that YouTube notifications might be triggered by a Client Side query, meaning the code when you refresh or open a new page says “Hey YouTube, can you look up my list of notifications and tell me if there are any new ones?”, which spreads the load of the notifications around over time because people are refreshing at different times (aka YouTube doesn’t know about the notification until you ‘ask for it).
Discord could be a Server Side push, since it’s built around being more of a real time communication system, so the Server has to look up each Client that is active, and tell them about the notification the moment it happens (which is much harder to split between multiple Servers and make sure nobody gets missed or nobody gets the same notification twice).
The difference is that YouTube is effectively a Client side Pull, and Discord is a Server side Push. Client side Pulls might be a lot of work for the Server to determine each Pull individually, but it’s easy to spread out over time and much easier to have multiple Servers that can answer questions about relatively static data.
Again totally speculative, because we don’t know their code.
Google has a top notch Cloud Computing Service called Google Cloud Platform (GCP). There are many services that can be used to manage this kind of needs. Like ”Pub/Sub” which can hold a message until it’s delivered with a throughput of 10k messages per second. Google has probably use its own plateform or something stronger so each video / YouTube creator is marked as a ”PUBlisher” and watchers are marked as ”SUBscriber”. Once a new video is created, the publisher generate a message that will be delivered to every subscribers.
The real answer is optimization.
The same basic principle is the same, send out notifications, but that can be optimized, as others have mentioned a pub/sub model where they keep a list of the topics and who subscribes to them, and then send out messages, but it can also be optimized further.
Break those jobs up into smaller batches and have multiple servers send out those notifications, making something that 1 server would take 10 seconds, and have 10 servers do a 1 second job at the same time.
Things can be taken even further, and when it comes down to it, Youtube is much larger than discord so they can afford more servers to break those jobs into smaller and smaller batches, but also they have more money to spend on manpower for better optimization.
Latest Answers