You can’t. Unless your entire systems supports hot swapping and has redundencies. For instance servers usually have hard drives that are hot swappable, meaning can be replaced while the machine is still running. Some external GPUs I believe can also be hot swapped, I have yet to hear about CPUs and RAMs though.
Edit: to answer your second question: We usually design software in such way that it can either be safely stopped and restarted or more commonly use the divide and conquer method and write software that works with smaller chunks of data to add to bigger collective and can therefore be stopped before proceeding to solve the next problem.
Latest Answers