I’ve heard this sentiment a fair bit in recent years and I’ve never been able to find a good answer online.
In: Technology
Suppose you need to interact briefly with people who speak a language you don’t understand. Let’s say you’re “undercover,” and you don’t want them to suspect that you aren’t one of them. You want to appear to belong.
One way to do that would be to learn the language.
We have a mental model of the world, and when someone tells us something, we add (maybe temporarily, maybe permanently) some new pieces to that model. We know how a lot about how the model behaves — what is possible and what isn’t — so we can reason based on it. Learning a language means learning how to translate sentences in that language into our own mental model. When we respond to what someone says, we translate some information about our (updated) model into words.
No one (yet) knows how to do that with a computer. Computers aren’t even close to being able to model the world as humans experience it. It’s not just that we don’t know how to “translate” language: there is nothing into which to translate it that could do the job of representing the world in the sense that we represent it in our minds.
So if learning the language isn’t an option, what else could you do? You’d observe common interactions. Someone says, “Hellozon der,” and the response is almost always, “Hidoyuh too!” When a person says, “Houzay bee-in hangeen?” the answer is usually “Goodz gold, danken fer asken!” You might be able to memorize enough common phrases and responses to fake your way through. You might even start to get a little sense of context — maybe, after one minute or more of conversation, if someone says, “Hellozon der,” the response is, “Gooden hellozen, morrow zeeya,” and it’s time to walk away.
While computer programmers don’t know how to make a computer “understand” anything the way humans do, rapid, systematic processing of massive amounts of data — even millions of times what any one human being could manage — is what computers do very well. What current AI does is like observing common phrases and responses — but far more of them that any human could, using existing data, like Reddit posts and responses — and tabulating the connections to create a “large language model.” Then it searches for patterns in the input and computes the most likely output.
No can give a satisfying answer to “how it works” because it doesn’t work the way it appears to work. Like the undercover agent, it appears to understand the language and give meaningful responses, but it doesn’t. It just uses a staggering amount of data — and some very sophisticated statistical analysis — to make a really good guess about what output you would expect. A literal accounting of “how it works” on any given input would run to millions of lines, but it still wouldn’t tell you anything you cared to know, because it’s just the process of making a “guess” based on tabulated statistics. At no point does it “understand” anything, or “draw conclusions” in the sense that a thinking human being does.
It’s not that they don’t understand how they work, only that they don’t know every line of code like a traditional program.
AI like this learns. It’s fed millions of documents or images and told what each is or something like that. Eventually, it can recognize what an apple looks like etc. This only happens because it can change it’s own programming. Because of this, it’s impossible for anyone to be able to predict exactly what it will do, or exactly what steps it will take to accomplish a task.
The original code was written by humans, but what the AI becomes has parts that have never been touched by a person.
Easiest way to properly explain it, is by explaining genetic algorithms first.
Imagine you take an electrical circuit, and just throw electric components onto it *completely randomly*. It’s a complete meaningless mess that makes no sense. You send electricity to it and you see the output on one of the wires – the circuit probably explodes or does nothing. This is fine.
You repeat the same process ten, hundred, thousand times. Finally you find a random combination of components that has some sort of output – 2 volts on the other end. You designate this circuit a successful one. Even though your goal is to get 10v output.
You take the successful one, and randomly replace few components on it. You do this a million times with the same “parent”, this creating a million slightly different “offsprings”. You try all of them out.
Many burn out, some do nothing, but there are a few that produce a number closer to the 10v you are looking for. Some might be 6, other 3, other 90.
You pick the closest one (6), designate it as parent, and repeat the process again.
You continue doing this until you get an answer with 10v.
Now, your circuit will be stupidly complicated, it will have a weird mess of things, paths that are isolated and never powered, connections that make no sense at all – but in the end it gives you the right answer.
This is the fundamental idea behind modern AI (neural networks). It’s very different too – but fundamentally you have the same garbled mess that somehow works and somehow produces the right answer.
It means the network started producing some results they did not expect and cannot completely understand.
Artificial neural networks of non-trivial size are impossible for us to completely understand once fully trained such that we can predict all that they can do.
AIs like ChatGPT and MidJourney are types of Neural Networks.
Neural networks are made of one part code and one million parts training data, which is then compressed into a “model” which is somewhere in the middle in terms of size.
Many people understand the code, but it is much harder to understand the models, mostly due to their sheer size, and also because they change so often, either due to tweaks in the code or introduction of new datasets.
Unlike traditional programming, which was more based on control over the machine, machine learning approaches are not concerned with controlling or understanding the machine, rather they are concerned with achieving results by supplying massive amounts of computational energy.
Latest Answers