A rocket has to go really fast to reach speeds fast enough to orbit, and the atmosphere is thickest close to the ground. So a rocket tries to get out of the thickest part as fast as it can to save fuel, then it turns sideways to get to orbital speeds. Basically it would waste too much fuel going through the atmosphere like a plane.
Latest Answers