why do models like ChatGPT forget things during conversations or make things up that are not true?
In: 803
There’s an old thought experiment called “The Chinese Room.” In it, there is a person who sits in a closed off room with a slot in the door. That person only speaks English, but they are given a magical book that contains every possible Chinese phrase, and an appropriate response to said phrase also in Chinese. The person is to receive messages in Chinese through the slot in the door, write the appropriate response, and pass the message back through the slot. To anyone passing messages in, the person on the inside would be indistinguishable from someone who was fluent in Chinese, even though they dont actually understand a single word of it.
ChatGPT and other LLMs (Large Language Models) are essentially that. It doesn’t actually _understand_ what it’s saying, it just has a “magic translator book,” that says things like “if I receive these words next to each other, respond with these words,” and “if I already said this word, there’s a 50% chance I should put this word after it.” This makes it really likely that when it rolls the dice on what it’s going to say, the words work well together, but the concept itself might be completely made up.
In order to “remember,” things, it basically has to re-process everything that was already said in order to give the appropriate response. LLMs have a limit to how much they can process at once, and since what’s already been said is constantly getting longer, eventually it gets too long to go that too far back.
Latest Answers