why do models like ChatGPT forget things during conversations or make things up that are not true?
In: 803
So, I see a confidently wrong answer here: that it doesn’t “understand”.
It absolutely develops understandings of relationships between words according to their structure and usage.
Rather, AI as it stands today has “limited context”, the same way humans do. If I were to say a bunch of stuff you you that you don’t end up paying attention to well, and then I talked about something else, how much would you really remember of the dialogue?
As it is, as a human, this same event happens to me.
It has nothing to do about what is or is not understood of the contents, but simply an inability to pay attention to too much stuff all at the same time. Eventually new stuff in the buffer pushes out the old stuff.
Sometimes you might write it on a piece of paper to study later (do training on), but the fact is that I don’t remember a single thing about what I did two days ago. A week ago? LOL.
Really it forgets stuff because nothing can remember everything indefinitely forever except very rare people and the people that do actually remember everything would not recommend the activities they are compelled to engage in that allow their recall: it actually damages their ability to look at information contextually, just like you can’t take a “leisurely sip” from a firehose.
As to making things up that aren’t true, we trained it explicitly, tuned it, built it’s very base model, from a dataset in which all presented response to all queries was confidently providing an answer, so the way the LLM understands questions is “something that must be answered as a confident AI assistant who knows the answer would”.
If the requirement was to reflect uncertainty as is warranted, I expect many people would be dissatisfied with the output since AI would render many answers with uncertainty even when humans are confident the answer must be rendered and known by the LLM… Even when the answer may not actually be so accessible or accurate.
The result here is that we trained something that is more ready to lie than to invite the thing that has “always” happened before when the LLM produces bad answers (backpropagation stimulus).
Latest Answers