Less fuel is used going straight up. Planes can reach outer space if their engine doesn’t rely on air (i.e. a rocket motor). The X-15 was able to reach space, albeit briefly.
But it’s not just reaching space, it’s also reaching orbit, and for that you need two things, speed and altitude. Going straight up initially then pitching over is the most fuel efficient way of getting large payloads into orbit.
Latest Answers