What is Overfitting in machine learning and why is it bad?
In: 2
Say you sample a sine function, but your data points have some noise. Now say you attempt to fit a polynomial of degree N to those data points i.e. a + bx + cx^2 +… zx^N, assigning values to the coefficients to minimize your error.
If you let N=1 then you can only make a line, so not a good fit. Let N=2 and you can make a parabola, which is closer. If you continue to increase N you get a more and more complicated curve which gets closer and closer to every data point. Eventually N becomes large enough that your function exactly matches all data points with errors of zero, but the problem is that you now have a crazy looking squiggly line that no longer reassembles the smooth sine function which generated the data. Thats because you gave your function so many degrees of freedom that it was able to exactly fit the noise rather than average the data like it would have if it had fewer parameters to work with.
Latest Answers