What is Overfitting in machine learning and why is it bad?
In: 2
In very simple terms, overfitting is when a model appears to work very well on one datatset, but it completely breaks down on another one. Usually this means that it performs well on the training and validation sets used during development, but it doesn’t actually work when it’s given data that it’s actually supposed to process in practice. That’s why it’s bad: it ends up being useless.
What overfitting actually is depends on the model, but in general it means that the model has learned to exploit some peculiarity of the training dataset that is not present “in the wild”. For example, if you were training a model to look at pictures of people and tell you wether they have blue eyes or not, and every single blue eyed person in your training datatset had blonde hair, the model could learn to actually recognize blond hair. Then if you gave it a picture of a brown haired, blue eyed person, it would tell you that they don’t have blue eyes.
Latest Answers