I’m reading Richard Powers’ *Galatea 2.2*, >!which tells the story of an engineer and a writer developing a computer-based neural network capable of taking as input a piece of English literature (e.g. a sonnet) and outputting analysis equivalent of a 22-year-old’s level!<. **In the book,** ***weight/ed vectors*** **get multiple mentions and the concept has me stumped.**
In: 0
There are two relevant things here.
First: what’s a vector? It’s a mathematical object that can have multiple independent components.
For a physical example, take something’s position in space. That can be described by the vector <X,Y,Z>.
But that’s not the only thing that can be described as a vector. The input to a neural network can be represented as a vector.
It’ll be easier to understand this if we look at an image instead of text. Particularly, a black-and-white image.
To describe this in a way a computer could work with it, we can break that picture down into an array of pixels. Each pixel has a color value, and each color value can vary completely independently of any other color value. That makes it a vector.
That leads into the second thing: how a neural network actually works. Neural networks (at least, some types) are formed of “layers” of “neurons.” A neuron is a node of the network that can hold a particular numerical value. Several to many of these are arranged logically into layers; typically, every node of one layer connects to every node of the next layer. The nodes and the connections between them have something called a “bias” and something called a “weight.” The nodes also have some threshold value that I’ll explain below.
A “bias” is like a buff or debuff. It adds or removes a certain amount from the numerical value the node holds. A “weight” is like a scale factor. Something with a weight of 2, for example, will typically be twice as influential as something with a weight of 1.
When a signal reaches a layer of a network, the weights and biases are used to calculate the signal each node sends to the next layer. If all of the inputs to a given node (accounting for all the buffs, debuffs, and scale factors) surpass that node’s threshold value, the signal is transmitted along to the next layer.
Eventually, there’s an output layer that doesn’t connect to a further layer. Back to our example of a black-and-white photo… say the neural net in question is designed to apply a Picasso style to a photograph. The input vector would be the color values of all of the pixels in the input image; the output vector would be the color values of all the pixels in the output image. And for every layer along the way, there’s a “weighted vector” that describes the state of the network as the signal propagates through.
Latest Answers