What suddenly existed that allowed these breakthroughs to occur that did not exist before? What made technology conducive to Al breakthroughs that could not have happened years ago? Was it faster processors, smarter programming from the coding community or something else? What suddenly happened that permitted these breakthroughs to happen?
In: Technology
Two factors.
The first and biggest is the publication of the[ transformer model](https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture)) paper. This paper was published in 2017. The recent AI technology breakthroughs are variants of this model.
Not long after, researchers noticed that transformer models scaled extremely well with training data compared to other models. They also noticed that it required and responded well to human feedback. Human feedback is needed to steer these networks to a workable state. It took a few years to build the tooling needed to both feed internet scale training data and set up large teams of humans to provide reinforcement learning from feedback.
Latest Answers