Parallel systems do the same thing in parallel on several computers. This of course requires a problem that can be divided into equal segments.
A distributed systems do different things on several computers. They can be more versatile, and so it’s the general solution for most compute intensive problems.
On a parallel system, you can generally assume that all data can be accessed by all “threads” with little cost. This could be 1 system that has a bunch of cores and the data stored in ram, or it could be a networked cluster with a high performance centralized file system. You can also generally assume failures are rare and not account for them.
On a distributed system you can not assume this. The data may be extremely expensive to move between systems, and each may need to operate independently. The data may not even exists in 1 central location, and it may be impossible get the current state of the entire system’s data with complete accuracy. You generally must also assume that any part could fail or disappear at any time and keep running anyway.
Latest Answers