This goes for almost all AI language models that I’ve used.
I ask it a question, and instead of giving me a paragraph instantly, it generates a response word by word, sometimes sticking on a word for a second or two. Why can’t it just paste the entire answer straight away?
In: Technology
It’s just not fast enough to give the whole answer straight away; getting the LLM to give you one ‘word’ at a time is called “streaming”, and in some cases it is something you have to deliberately turn on, otherwise you’d just be sitting there looking at a blank space for a minute before the whole paragraph just pops out.
Latest Answers