It’s mostly a combination of speed and bullet design. Faster bullets tend to have a lot more energy because energy is proportional to the velocity squared. Rifle bullets can go 2-3x faster than a handgun, which means 4-10x the amount of energy.
With bullet design, rifle bullets tend to be pointy and long, this is called a spitzer bullet, handgun bullets are more blunt and short and fat. With a long pointy bullet, when it hits the kevlar they put their energy into a smaller area on less fibers, making it more likely the kevlar will fail. You can see it action if you look up 300 blackout, it shoots a spitzer bullet at the same speeds as a handgun, and it can penetrate kevlar vests just because of the long pointy bullet.
Latest Answers