Today, Google announced the release of Gemini 1.5 Pro, its next gen LLM.
Sundar Pichai posted: “Gemini 1.5 Pro, our mid-sized model, will soon come standard with a 128K-token context window, but starting today, developers + customers can sign up for the limited Private Preview to try out 1.5 Pro with a groundbreaking and experimental 1 million token context window!”
What does it mean to have 1 million token context window and how does it compare with the previous Gemini 1.0 Pro and OpenAI’s GPT 4.0?
In: Technology
“Context window” is basically the model’s memory–it’s how much of what you’ve previously input into the model that will affect its results for your next input.
“Tokens” get a little weirder. They’re essentially pieces of meaning. That can be sentences, words, individual letters–it really depends on the model and on how it processes input. When you input something into the model, it breaks it down into appropriately sized chunks of meaning (as another commenter mentioned, “red” being one token, vs “r” “e” and “d” each being a token).
This particular model has a context window a million tokens wide, which means it can remember what you’ve said to it up to 1 million tokens.
As a quick example, pretend the model has no context window at all (or one only as big as the size of the input you’ve just made to it). You say “this is Bob, and he’s a vampire.”
Next, you ask it “what is Bob?” And it’ll spit out something about Bob being a typical human name, because it has no memory of you telling it Bob is a vampire.
Now, the same model but with a context window: you tell it Bob is a vampire, and then when you go ask it what Bob is, it will remember that Bob is a vampire (as long as you didn’t tell it that too long ago). The width of the context window determines what “too long” is.
ChatGPT has an 8,000 token context window (and I think an upgraded version with 32,000?). This means that Gemini’s able to ‘remember’ things for significantly longer than ChatGPT.
Latest Answers