why do models like ChatGPT forget things during conversations or make things up that are not true?
In: 803
ChatGPT is basically a text predictor: you feed it some words (whole conversation, both user’s words and what ChatGPT has responded previously) and it guesses one next word. Repeat it a few times until you get a response and then send it to user.
The goal of its guessing is to sound “natural” – more precisely: similar to what people write. “Truth” is not an explicit target here. Of course, to not speak gibberish it learned and repeats many true facts, but if you wander outside of its knowledge (or confuse it with your question), ChatGPT gonna make up things out of thin air – they still sound kinda “natural” and fitting into the conversation, which is the primary goal.
The second reason is the data it was trained on. ChatGPT is a Large Language Model, and they require really *huge* amount of data for training. OpenAI (the company which make ChatGPT) used everything they could get their hand on: millions of books, Wikipedia, text scraped from the internet, etc etc. Apparently important part was Reddit comments! The data wasn’t fact checked, there was way too much of it, so ChatGPT learned many stupid thing people write. It is actually surprising it sounds reasonably most of the time.
The last thing to mention is the “context length”: there is a technical limit on the amount of previous words in a conversation you can feed it for predicting next word – if you go above, the earliest ones will not be taken into account at all, which seems as ChatGPT forgot something. This limit is about 3000 words, but some of it (maybe a lot, we don’t know) is taken by initial instructions (like “be helpful” or “respond succinctly” – again, a guess, actual thing is secret). Also, even below context length limit, the model probably pays more attention to recent words than older ones.
Latest Answers