Why in the Ordinary Last Squares method are the distances from the line measured straight up/down?



Why does the OLS method minimize the distances from the estimated line that are parallel to the Y axis?

Shouldn’t the distances be minimal if they were at 90° angle to the estimated line?

In: 3

The x value is fixed. It’s the input variable for your model. The y value is what is being predicted. You are measuring the error between the predicted value and the known value.

Here, “distance” means the difference between the observed value and the estimated value, not the length of the shortest line segment that passes through the point and intersects the line. The latter definition is fine for geometry, but the former is more useful for modeling.

Generally you assume that you have no error (or at least very very little error) in the x-position, and larger error in the y-direction. This means that its super unlikely a point will be to the left/right of a point on your model. It’s much more likely to be directly above/below, so you fit based off of the y-error.

For cases where you actually do have uncertainty in the x position as well, there exist other fitting methods, like Orthogonal Distance Regression that do more like what you’re asking about.