Distributed Computing

In: Technology

You know how cleaning your room is a big job, but if 3 friends came round and one tidied clothes, one dusted shelves, one put books away, one changed the bed and vacuumed how it would be quicker?

By dividing a big job into multiple smaller jobs that could be done at the same time, you can get more done quicker. However, like all working in the same room, some jobs are faster than others, and you may get in the way of one another, or be waiting for one another to do something, you need to plan carefully to get the best effect.

Let’s take an example that could be done in a classroom.

One way that we calculate the value of Pi is with something called the Leibniz formula. It’s an infinite series that goes something like…

1/1 – 1/3 + 1/5 – 1/7 + 1/9 – 1/11 + 1/13 … = pi/4

If you calculate that by hand, you can keep calculating more and more terms for it, and keep getting closer to the actual value of pi.

Now imagine you split this up to a bunch of kids in a class room. You give each one of 20 students a fraction to expand. They expand those and bring them back to you. You then take those values and add/subtract them together yourself to figure out the value of Pi.

You needed to solve all those fractions, but you gave the work out to each of the students. They did the work and returned it to you, and you then did something with that work.

Distributed computing is similar. You have a problem that can be broken down into smaller components. Each worker gets a small component, and then you coordinate when each one gets done, and take the results back to do something with. Often times those results will then result in more work being given out, which makes this a more continuous process.

Ok, so let’s say I’m a scientist, and I want to do some really big math problem.

I might start, by writing a program to solve it on one CPU. But it takes too long.

So I split the problem up into multiple threads – that’s multiple independent sequences of work that can run in parallel – and run it on all the CPU’s in one computer. But that still takes a while.

But, there are other scientists, and people who like my work and want to help me, who have their own CPU’s lying around doing nothing. More CPU’s than I could ever dream of owning. And these people would gladly use their CPU’s to help me.

So, I write a program, to split the problem up into smaller pieces, send them over the internet, and collect the results. And I write another program, to gather small pieces of the work, do it, and send the results back over the internet. And these people download the program and run it.

And this way, I get the work done faster than I could have otherwise, and the people whose CPU’s I borrowed are just happy their computers could do something useful.

That’s what distributed computing is – splitting up some computational work, sending the little pieces of work over the internet, and collecting the results.

I’ll use [Distributed.net](https://Distributed.net) as an example. They were trying to show that it was feasible to break common encryption of the time. Encryption has a key space that you need to crunch through, and it’s far more than one computer could do in any reasonable amount of time. But this key space could be divided up as needed. So people installed a client on their computers. This client would go out to [Distibuted.net](https://Distibuted.net), grab a chunk of key space, and set your computer to crunching it. When it was done it would return the results and get another chunk.

This may not sound like much, but millions of people doing this with their computers can crunch through that key space rather quickly.

This was also done by [email protected] to do calculations on protein folding. And with that the PS3 game console was actually one of the top contributors for a while.

Sending off math problems to your friends in order to solve a problem set faster.

Note here that it’s important that we don’t waste any effort. If I mail my friend a letter to ask her what 1+1 is, I’m wasting time, I can do it myself. The letter takes longer to send than it does to solve the problem.

But say I had something like, “I’m giving you 100 problems and Jeff 100 problems, write down the answers and then bring it back.”

They each can work on their 100 problems, then return the letter by the time a single person got to maybe 150 problems. Now we’re saving a lot of time and we just split it into two.

Imagine I have dozens of computers, or hundreds, and I can split my questions so that they can all work together to get the answer.

Imagine it takes a computer 10 minutes to add 2 numbers and you need to solve:

1+2+3+4+5+6+7+8+9+10

It would take this hypothetical ridiculously slow computer 90 minutes to solve this problem.

You could use 2 computers instead. Have one computer solve 1+2+3+4+5 and the other computer solve 6+7+8+9+10 then add the result from both computers together, it now takes 55 minutes to solve the problem (including the final addition of combining the two results), add more computers and it gets even faster.

Instead of having one computer solve a million math problems, use a million computers to solve one math problem each. That way all million problems are solved in the time that it takes to solve **one** problem.

It takes a little extra time to split the work up and then compile it all back together again, but it’s still an enormous time savings.