What does it mean when people say that not even the creators of large neural networks or AIs like ChatGPT or Midjourney fully understand how they work?

Question

I’ve heard this sentiment a fair bit in recent years and I’ve never been able to find a good answer online.

In: Technology

Answer 1

This is an important distinction: it’s not that we don’t understand the principles of how they work — they were designed, after all. The issue is that the system is so complex, consisting of billions of paths that form a ‘neural network’, that nobody can tell exactly which path a given input takes to end up in a given output.

This is also a very real problem in that for example a recommendation cannot be validated when it’s not possible to know which data points weighted it in which ways. There are performance reasons, but it’s quite possible this is an intentional decision to avoid confirming that copyrighted or otherwise disallowed material was unlawfully used to train the model.

There are some more excitable people and charlatans who claim that there is some kind of an ’emergent consciousness’ inside LLMs that’s making decisions independently, but that’s just nonsense. All we have is an unimaginably large number of data points.