My very limited understanding of GPT is that it’s basically a text generator. Why and how could it solve logic and math problems? Or is it just an emergent ability from LLM that nobody understands?
In: 4
Yes GPT is an advanced generator designed to predict the next “symbol”, but that doesn’t mean that it can’t “learn” underlying principles.
It is certainly able to answer simple math problems that it hasn’t seen before, so in some sense has “figured out” the basic principles of, say, addition, because it’s seen enough examples that it can generalise.
This doesn’t mean it’s going to get everything correct though. I watched a video from the creator where he said that it had “learned” to add any two 40 digit numbers together, but if you give it a 35 digit and a 40 digit it would sometimes (confidently) get it wrong.
Of course, a human might know that they are bad at that sort of thing (we also make mistakes), but knows enough to use a calculator. With the Wolfram etc. plug-ins, this is exactly what future versions will do.
“ChatGPT can add two 40 digit numbers, so now it “mostly” understands how to add, but if you try a 40 digit number plus a 35 digit number, sometimes it gets it wrong. So it’s still deriving how math works..” Greg Brockman, founder of #OpenAI at #TED2023
EDIT: Added quote
Latest Answers