– What are the differences between the main types of variables within different LLM?

Question

I starting messing around with LLM and the amount of information is impressive to say the least… I’m trying to figure out basic python basics skills and currently I’m having some victories to make stuff work.

But while trying to choose the best models for my the project objetive and that my system can handle (RTX 4070, Ryzen 7 7000X, 32GB DDR5, Windows 11), I’m kind of overwhelmed by the amount fo information and variations each model may have, and these differences are usually represented my letters and numbers in the model’s name… Here are some examples of variables that I encountered:

– GGUF, AWD or GPTQ

– Q2, Q3, Q4…Q6 or Q8

– \_0 or \_K

– \_S, \_M or \_L

– dpo

– laser

I’ve been reading about each of these differences, but I think I lack the basics, because while reading about it everything makes sense, I’m having a hard time to understand well enough to avaliate these difference while choosing a model.

What are the main differences between these variables and how should I use these differences to choose a model well adapted to my needs and to my system?

In: Technology

Answer 1

I think, if this is your first foray into coding, that you should be more concerned with sticking to it rather than the technical concerns your post is about. The hardware used when starting coding, and tackling the projects you will use to learn, is not really relevant.