They don’t.
They orbit in roughly the same plane because they started as a cloud of dust and gas and then spent billions of years bumping (gravitationally) into one another.
In three dimensional space, clouds of gas and dust always have an average angular momentum that rotates around one axes, so having it bump into itself for long enough will form a disk.
Because of conservation of angular momentum.
Conservation of angular momentum basically says that, if something is spinning in a direction, it will keep doing so.
When the solar system was a big cloud of gas and dust everything was spinning in all different directions. However, **on average** across all that gas and dust there was a particular direction things were spinning. That is, if you cancel out all the spin of all the objects spinning in opposing directions, you end up with a single direction of spin.
That “cancelling out” was exactly what happened as all that gas and dust collided as the solar system condensed. What we’re left with are several large and many small objects that are, by and large, all spinning in the direction of that overall average spin of the early solar system.
True ELI5:
So, before the solar system was here there was a big cloud of little bits of stuff. This cloud got thicker and more compact in the center, starting to turn a bit while more stuff “fell” into it. As it got more and more dense and bigger, it stared bending the fabric of space more so that the rest of the cloud began spinning around it, and overall spinning in the same direction.
When the cloud starts spinning like that everything wants to settle together in the same overall plane around the center of the large spinning object. This can be easily seen with 2 objects in our solar system in the middle of an early formation: Saturn and Uranus. Both have ring systems that are across the equator of their individual spins. In the case of Uranus, that spin is closer to a 90° from the sun’s spin/Solar plane; so the rings are also 90(ish)° to solar.
It all started as a supernova, an exploding star that made a huge cloud of debris. As it blew out, gravity between clumps in the debris cloud pulled them towards each other, and pulled more in, which started a circular spin in the debris that was glomming together from its gravity. And it pulled in more of the debris and spun faster and a lot pulled right to the middle and eventually piled up and ignited as the sun. A lot of the rest of the spinning cloud spun faster and flatter and couldn’t fall into the middle, and its own lumps started glomming together the stuff left in their area of the spinning cloud. All those gravity clusters started their own spins, but were all still spinning around the Sun, now we’d saying orbiting, as part of that original big spinning debris cloud it all formed from. The whole solar system is just the collapsed lumps of a spinning supernova debris cloud, evened out into a single common plane from the original spinning and the planets’ gravity.
You have the problem reversed (should be “why aren’t electrons arranged like a planetary system?”). A fluid, when rotated, will see material migrate outward from the center and flattening of the sphere into a disc (the distance along the axis shortens as material migrates outward near the equator). The fastest motion is occurring at the equator (most distance traveled per unit time the further away from the axis of rotation the matter is located) so that stuff is impelled to move outward even further. The slowest motion is happening at the axis (zero displacement at the very axis), so that material is “pulled” parallel to the axis and toward the equator in order to replace the lost materials in that region.
Liquids, gases, and semi-fluid mixtures will all do this sort of thing, whether water on a spinning plate or clay on a potter’s wheel. The main concern is what other forces or processes are affecting the way the fluid responds over time to that push to spread outward along the equator and shorten the axis of rotation?
With planetary systems, the center is a strong gravitational attractor, which is what keeps the disc from becoming depleted in the center zones with time. When gravitational attraction is powerful, the outward movement is largely offset by the pull back toward center. You end up with an “oblate spheroid” like the earth, where the diameter at the equator is longer than the diameter along the axis, by some percent depending on rate of rotation (with no rotation, there is no outward force to offset and the distance from center of mass will be the same at surface everywhere, because gravitational attraction is an inverse function of distance ONLY in that situation).
The sphere is flattened, and it is just a matter of how much “cohesiveness” keeps it from distorting into a disc rather than a flat-ended sphere (a disc is basically just a sphere which really flattened out). The solar system is pretty diffuse (not much mass per unit volume and no other important forces acting at the scale of the system) so gravity is not driving the retention of a spherical shape.
In atoms, there are multiple forces involved, with electronic (charge) attraction and repulsion dominating. This forces the electrons into movement patterns that maximize the attraction toward the nucleus while minimizing repulsion by interaction with nearby passing electrons. You end up with a cloud of electrons, with each electron traveling in high-frequency zones and not drifting (much) into other regions occupied by a different electron where that region is its high-frequency zone.
The mathematical description of the various forces that come into play is somewhat complex, but the solutions to the applicable equations indicates these oddly shaped “orbitals” or “electron paths” provide the minimum energy conditions (least repulsion or other interactions with maximum attraction to keep the electrons in the zone of influence of the positively-charged nucleus). Spinning rate (displacement speed) is actually not variable (unlike with planets, solar systems, water on a plate, clay on a potter’s wheel), given that electrons are basically all moving at the speed of light wherever they exist in the “sphere” of the atom. The rotation of the atom is not what is driving the movement.
We do see evidence that the predicted electron zones are actually and truly occupied, via the way that bonds form between different elements and other things. But we really do not “see” where the electrons are truly located specifically. We calculate that distribution. It is not a “popular” depiction. It is what we believe is a good way to represent the probability distribution for the location of given electrons. Most people only know the basic structure expected for a low-number element anyway (first two electrons of the S occupy a spherical zone and the next 6 p electrons are located as “dumbells” along the x-y-z axes. The weird donuts and disconnected spots of higher number electrons are generally not known.
What other people said about angular momentum, but also, there is an additional small effect where planets will tend to pull other planets into similar orbits over time. When two planets orbit close to each other, they exert a very small pull which bends the orbits slightly towards each other over time.
Latest Answers