why do models like ChatGPT forget things during conversations or make things up that are not true?
In: 803
It does not give answers.
Re-frame your thinking this way: It gives you text that is supposed to look like something a human could give you as response to your input (question). It just so happens that the text it finds most related to your input tends to be what you’re looking for and would consider to be the “right answer”.
The following is not how it works in reality, but should help you understand how these language models work in general:
The AI takes the words in your input, and searches in what context they have been used before, to determine the associations. For example, it can figure out that when you ask about sheep, it will associate with animal, farming and food.
So it then searches for associated text that is the best associated with all those meanings.
Then it searches for the most common formatting of presenting such text.
Then it rewrites the text it found tho be best associated, using formatting (and wording) of such text.
At any point it time it actually understands what it is saying. All it understands that words sheep, farming and animal are associated with an article it found that discusses planting (because farming), farm (animal). So it gives you that information re-formulated in a way suitable for text.
That’s why if you ask it “How deep do you plant sheep?” it might actually answer you that it depends on the kind of sheep and the quality of soil, but usually about 6 inches.
Again. Please note that is is not actually what happens. Whether there are any such distinct steps is something only the AI creators know. But the method of association is very real, and very used. That’s the “Deep Learning” or “[Neural Networks](https://www.youtube.com/watch?v=aircAruvnKk)” that everyone talks about, when they discuss AI.
Latest Answers