Like the title says, how have we gone from ChatGPT being the apex of it’s type (and therefore presumably very complex and rare) to seeing so many clones in sl short a time? Wouldn’t the code/ system that makes up ChatGPT be in copyright or the code be difficult to mimic, wouldn’t the amount of data scraping take forever?
In: Technology
> Wouldn’t the code/ system that makes up ChatGPT be in copyright or the code be difficult to mimic
The code is surprisingly simple. OpenAI is a non-profit research group (now with a for-profit arm), and they’ve [published](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf) all the initial research that went into GPT. Nowadays you can create an LLM in just a [couple hundreds of lines of code](https://github.com/karpathy/nanoGPT/blob/master/train.py).
What’s prohibitive about creating them is the training data and compute resources. GPT 4 cost more than $100 million to train. Getting models that high in quality is really only doable with Microsoft or Google money, but everything’s accessible enough that *anytime* can create something that works *reasonably* well for specific uses.
Latest Answers