Math behind the position of the point in linear regression with respect to the line?

Question

I was trying to understand how linear regression works from scratch. I found a video on linear regression explained by Luis Serrano. From his explanation, I found that a line and y-axis divide the 2D coordinate system into four sections. And there are four cases in which a line will rotate and translate to get to the near to the points.

Case 1: If the point is in the 1st quadrant above the line

Slope will increase =  anti-clockwise rotation and
y-intercept will increase = translation upward

Case 2: If the point is in the 2nd quadrant above the line

Slope will decrease =  clockwise rotation and
y-intercept will increase = translation upward

Case 3: If the point is in the 3rd quadrant below the line

Slope will increase =  anti-clockwise rotation and
y-intercept will decrease = translation downward

Case 4: If point is in the 4th quadrant below the line

Slope will decrease =  clockwise rotation and
y-intercept will decrease = translation downward

My question is how to calculate the position of the point with respect to the straight line? I know how to find whether a point is above or below the line by comparing the predicted y value and the original y value with respect to the x value. And according to that, I can do line translation or increase/decrease y-intercept.

But, how can I find where point is, left or right of the y-axis? And according to that how can I relate with line rotation to increase or decrease slope?

Answer 1

This seems like an unnecessarily confusing way of talking about regression lines and doesn't seem to have much to do with how we actually calculate these things or the problem regression is trying to solve.

My advice is to forget about rotation and quadrants and think about it this way: you have a cloud of points in the x-y plane, characterizing the relationship between X and Y. We want to know how much we expect Y to increase or decrease if we increase X by one unit.

This means we just want to find the straight line that gives us the best estimate of Y for a given X value. ALL lines are characterized by an intercept and a slope, so we need to find those two values to identify the "best" line. The intercept tells us where the line hits the Y axis, which is important, but we mostly care about the slope since that's what tells us how much Y increases (or decreases) for each one unit increase in X. A positive slope means that as X increases Y increases. A LARGE positive slope means that as X increases Y increases a lot. A negative slope means that as X increases Y decreases. A slope of zero means that there is no relationship between the two variables at all. So what specific line (that is what values of intercept and slope) do the best job of summarizing this cloud?

The classic "OLS" approach to this problem is to find the line that minimizes the sum of the squared errors between the line's Y value and the actual Y values at each X value. In other words, draw a line through the cloud, and for each X value look at how "wrong" the prediction is (how far away the actual Y value is from the line), square all of those errors and add them up. Then find the line that has the smallest sum of squared errors. This sounds hard but we can figure it out with some matrix algebra. If you want to add another variable (Z) then you now have a 3 dimensional cloud of points in a CD space and you find the plane that best summarizes them (so you need two partial slopes), and so on.

At no point does "rotation," or dividing the space into quadrants come into this procedure and I'm not sure why it's supposed to be helpful...

Answer 2

If I understand you correctly: if your point has a positive X value, it means it is located on the right of the y-axis. If it's negative, it's on the left of the y-axis.

This way of classifying the X-Y space into four sections might give you a good intuition about how the regression works, but remember, we don't run the regression by literally checking which section each data point lands.