I’ve heard this sentiment a fair bit in recent years and I’ve never been able to find a good answer online.
In: Technology
OP, I’m going to modify TitansFrontRow’s analogy to better describe a neural network and why even the designers don’t know how it did it…
Imagine that 3 holes in the fence example. You ask the dog to run through a hole, and he does. Well now let’s imagine that you have 3 holes on your back fence, 3 holes on your fence to the neighbor on the right, and 3 holes on your fence to the neighbor on the left. Each of your neighbors have multiple holes in each of their fences in different directions, as do their neighbors, and their neighbors, and their neighbors. One day, you are walking your dog and you are on the opposite side of your block and you tell your dog “go home!” and he takes off running through a hole in one fence. Your wife is at home in the backyard and 30 seconds later she calls you to tell you the dog is in the backyard. You don’t know which route your dog took to get through all of the neighbor’s yards and which fences he ran through, you just know you told him to go home, and he got home.
That’s how neural networks work. There are many “layers” which can each contain varying amounts of “nodes”. With this analogy, the layers would be each yard and the nodes would be the holes in a fence. But in an LLM, there would be thousands of layers, each with tens of thousands of nodes. And each node in layer N will have a connection to every node in layer N-1 and a connection to every node in layer N+1, so there are literally hundreds of billions of paths a single input could take to return a single output. And at the end, you have no idea which path your input took through those millions of nodes to give you the output you got, you just know that’s what you got.
Latest Answers