This goes for almost all AI language models that I’ve used.
I ask it a question, and instead of giving me a paragraph instantly, it generates a response word by word, sometimes sticking on a word for a second or two. Why can’t it just paste the entire answer straight away?
In: Technology
A lot of these answers that you’re getting are incorrect.
You see responses appear “word by word” so that you can begin reading as quickly as possible. Because most chat wrappers don’t allow the AI to edit previously written words, it doesn’t make sense to force the user to wait until the entire response is written to actually see it.
It takes actual time for the response to be written. When the response slowly trickles in, you’re seeing in real time how long it takes for that response to be generated. Depending on which model you use, responses might appear to form complete paragraphs instantly. This is merely because those models run so quickly that you can’t perceive the amount of time it took to write.
But if you’re using something like GPT4, you see the response slowly trickle in because that’s literally how long it’s taking the AI to write it, and because right now ChatGPT isn’t allowed to edit words it’s already written, there is no point in waiting until it’s “done” before sending it over to you. Keep in mind that its lack of ability to edit words as it goes is an _implementation detail_ that will very likely start changing in future models.
Latest Answers