It’s a choice by the game company. There are ways to load the next area while you’re still playing, but doing this is often technically complicated and requires additional time and therefore money. It’s easier to just show a loading screen.
Games that invest in smarter loading often do it because it’s a key part of the user experience they want. Other games would prefer to spend that effort on other features and content.
Latest Answers