Like the title says, how have we gone from ChatGPT being the apex of it’s type (and therefore presumably very complex and rare) to seeing so many clones in sl short a time? Wouldn’t the code/ system that makes up ChatGPT be in copyright or the code be difficult to mimic, wouldn’t the amount of data scraping take forever?
In: Technology
NVidia GPUs are what powers the training of large language models (LLMs) like ChatGPT. Having enough very fast GPUs (many $10M’s worth) running in the same datacenter with an enough storage and training data (from a major search engine such as Bing or Google) is what has only recently made this possible. The code is not that complicated and is now mostly open source. It takes months running the code on all those GPUs with the huge set of training data to train a new model variant. Now that the training data and processing power are in place, it is relatively simple to train new models or additional variants of existing models. People just have to wait for the GPUs to be available to run the next model.
As GPUs become even faster this type of model will become more and more common. All large tech companies seem to be working in these now.
Latest Answers