What has changed in recent A.I. advancements that made it leap forward so much lately?

Question

With chatGPT and A.I. generated art and so on, it seems like A.I. is really getting some traction. What has changed that we couldn’t do that before?

In: Technology

Answer 1

The big breakthrough with ChatGPT specifically is the Transformer architecture which came from a paper written in 2017.

The reason why AI is getting good now is that they need to be really big with a lot of neurons, and they need to be trained on a lot of data (in this case basically the whole internet). Companies haven’t felt confident devoting the resources necessary until recently when smaller models started to show promise.

Also there was a debate on whether or not just making the current models bigger would yield better results or if we needed new ideas, and it turns out that almost magically AI does get better on its own if you just make it bigger to the point where we see today.