I tried runing llma2 on my local machine, the size of the model is roughly 4 gbs, and it runs offline.
It has so far answered all questions I asked, about diverse topics in computer science, literature, philosophy and biology. How is so much information stored in such a small size?
In: 8
Valid sentences written in English are not very information dense, about 1or 2 bits per letter (there’s a very good ELI5 on that here: https://what-if.xkcd.com/34/). So if you’ve got 4 * 2^ 30 bytes times 8 bytes per bit divided by about 5 letters per word and 1.5 bits per letter, you can fit about 500 million words (about 5,000 books) of English into 4GB if the information is stored in the absolute most efficient way possible. That’s quite a lot, especially if you do a good job of picking which bits of writing are the most important.
Latest Answers