What does it mean when people say that not even the creators of large neural networks or AIs like ChatGPT or Midjourney fully understand how they work?

Question

I’ve heard this sentiment a fair bit in recent years and I’ve never been able to find a good answer online.

In: Technology

Answer 1

These neural networks are so named because their design is inspired by the brains of living things, like humans and animals. They are basically incredibly complicated wiring with billions of connections between the various parts.

But nobody really “built” these brains straight up. They are trained, which is a process involving giving it what is considered good information and tweaking the wiring of the brain over time until it is producing the good information we want. Training can easily take weeks of time.

But after weeks of time and letting the computer tweak the wiring of the brain… how is a human supposed to open up that box of wiring and make any sense of it? Even the computer has no idea why it’s doing most of the wiring adjustments it did beyond “doing it this way produced a better result during training than the other things I tried”. After repeating that process an absurd number of times, you’re left with something that you can’t really explain *how* it works, but clearly it *does* work.