I’m pretty sure MMOs such as World of Warcraft at their core are big databases. How are these databases able to connect to and manage data from hundreds of thousands of concurrent players in a way that feels seamless?
In: Technology
The truth is they aren’t one big database, it’s many many different smaller databases/servers. MMOs run on servers where you are only interacting with a most a few hundred players at a time, not hundreds of thousands. This means each server only needs to work with those few hundred players, not every single player.
To a server, your character is just a few lines of information. the less information you need to send and receive, the lower the latency and the more you can fit on it. It only takes fractions of a second to send this amount of information.
Things like the UI and graphics are all processed at the client side meaning the game only has to deal with a string of information that is moving along a 2d or 3d grid and interacting with other lines of information. Something even a desktop PC can do many, many times per second without issue.
They don’t.
In a bit of a simple term you have this architecture:
A game client connects to a server, server keeps current state of the game in own memory, and sometimes write stuff to the database (e.g. writes your position every 10-20 seconds or on your logout, but immediately writes your money or inventory into db).
The database keeps track only of stuff that is important not to lose, so a lot of dynamic stuff like mob health or cooldowns are not written to it.
So then what you can do is split players into different servers, or _shards_. The exact flow is dependent on the game. For EVE Online, each star system is independent, and can be hosted on separate server, and the load balancer will move star systems between physical servers to optimize costs. But because players don’t shoot at each other across star systems – there is no need to share realtime data or state between them.
For Star Wars The old republic – each planet is own shard. One planet might even have multiple shards, and you are assigned to the one with lowest players when you fly to the planet.
So TLDR:
– split the persistent data and data that you can lose easily. Write to DB only important stuff
– split world into independent chunks that don’t interact with each other, host those chunks on different servers
Each area is like a standalone game, you only interact with the players in that area.
For chat, it’s like Xbox, PSN, Steam where it’s an entirely separate service that runs alongside the game.
It’s an appearance of having all players in the same world but in reality players are playing in small groups.
Data is only read from the database (disk) when they are first loaded, after that they are stored in memory which is a lot faster. High end servers can have terrabytes of memory, although that is usually not necessary. The world is split into many smaller area that are processed by smaller servers. When a character moves from one area to another, the data is transferred to another server handling that area. There are processes that run in the background to update data back into the database to make sure not too much data is lost if a server crashes. In fact there are databases which handles this in memory and disk data storing under the hood and the developer does not need to care about it.
Usually, games like this loads your character from DB when you login, and then game server handles character logic (i.e. character becomes real time object in game). When you logout (and probably periodically as well) your character is saved back to database.
Just like a singleplayer game is not saved to disk every frame, its same thing.
And modern database may handle a lot of data. For example, I work on ES database that has 160 000 000 records and ES can find record in <5s. While total number of players in WoW might seem a lot, databases can take much much more than that.
*cracks knuckles
Ok, as others have stated, an MMO isn’t 1 big server. They are multiple servers all handling different areas, or sometimes different tasks.
Commonly there is are seperate servers just for chat, another just for login, and then a bunch of smaller servers for each area in the game.
The individual zone servers are always crunching simple math. This monster moved here. That player got hit for x damage. They are simple math problems, but they are handling thousands at a time.
Chat is usually a seperate server because of filtering and logging. These operations are a tiny bit heavier than simple movement or combat equations, and chat has to be able to communicate with all the other servers as well in case one player whispers or shouts out to the whole game. So chat gets its own server.
Lastly is login. Login is separate because it does a LOT of heavy lifting, fairly slowly, and it is kept separate for security reasons too. Login must take the username/passwords of the players and authenticate them. Once that is done the login server goes to the database and requests all of that users data. Compared to the normal movement and combat network traffic normally sent between player and server, the login server is pulling hundreds of times more data at player login. Inventory, stats, location, buffs, pets, etc etc. And all of that is getting pulled from a SQL database, on a hard disk. So not only is this operation pulling way more data than normal, it is also pulling it from the slowest medium, physical disk. So the login server yanks all the newly logged in player data, bundles it all up into a player object and then passes that neat little package off to a world server and the player appears in the game.
There is also “interest management”. if Player A is 10 miles away from Player B, they don’t see each other, then there is no point in the server sending Player A the data on what Player B is doing. So the server isn’t sending everyone everyone else’s position and HP and whatnot, only the ones that matter.
Servers also run at a certain “tick rate”. Minecraft in particular runs 20 ticks per second. Meaning 20 times a second it calculates monster/player position, HP, actions, etc. 20 times a second for a server is pretty slow, so there is a fair amount of downtime between ticks. That downtime can be used for other operations, like batching player data for save operations to the database.
Modern SQL databases are multi threaded and can support dozens/hundreds of simultaneous connections. So having a few dozen zone servers all saving player data isn’t that big of a challenge to overcome.
This is a VERY dirty explanation of what is going on between player and server
In our game, the draw distance of other players is dependent on your ping. The lower your ping, the more players you can see up to a point and your video settings. A single server can quite easily track hundreds of thousands of player coordinates without much effort, the limitation really being the underlying network and number of simultaneous connections so there are proxies involved. Other persistent world data is stored at a slower pace on different servers. The game client itself connects to a server that is just a router to these other servers and then messages are sent to the router. For incoming data, there is a single stream that the game client consumes in real-time. Easy to draw on a whiteboard, a bit harder to code.
In the case of World of Warcraft, there is actually not that much data that needs to be persisted in the database at any one time. Your character’s level, position, gold. Even the position does not need to be saved that often. This is why you might find yourself somewhere in the past if you get disconnected and log back in. The biggest part by far is the inventory. And you actually don’t change inventory that often, so even multiplied by thousands it is still manageable by a database.
Blizzard has a few tech articles out where they go into detail on how the database is the bottleneck for the game, and that is the reason they deployed the login queues. The login queues would control influx of db transactions, and players would be let in as the db activity would allow it.
This is also why modern WoW servers can host 10x more players than at launch: the database servers have increased drastically in power since then.
Latest Answers