Why can some video games host 10s of players with no lag while others can’t seem to host a lag-free 1-on-1?
There are too many variables at play here. So I can’t give a solid “This is why” answer.
It can depend on what kind of game it is, what the server situation is like, and your internet connection.
Some game companies host their own servers, which are usually well maintained and are on high quality connections. Others allow players to host their servers, which can have questionable quality. Some games are P2P (Peer-to-peer), in which your console/PC connects directly to the other players console/PC, this usually works well, but can lag if one of the players has a poor connection.
In all instances, peek hours, when the most people are in-game, can usually result in lag due to high traffic.
It depends on what needs to be done. Some MMOs might only need to update client information at a rate of once every second or so, while first-person shooters generally need to update dozens of times a second, causing much heavier server load per player.
Different game engines also have an effect, as does the competence of the programmers. There are a pile of other variables in addition to these.
Without even trying to explain what causes lag, the short answer is that some programmers know what they are doing and some don’t.