Question
What would be the intuitive explanation for the slope of Ordinary Least Squares(OLS), which is $\frac{cov(X,Y)}{var(X)}$ contributes minimizing the sum of squared residuals?
In the same sense, why the intercept is decided to make the mean of the residuals 0 (not the mean of squared residuals, but just the mean of residuals) ? In other words, conditional mean of a response varialbe is at least right on average. What would be the intuitive explanation for this should be the case?
More details
- According to the Chapter 4 of The Effect: An Introduction to Research Design and Causality, OLS's slope and intercept are decided so that the sum or squared residuals (SSR) is as smallest as possible.
OLS picks the line that gives the lowest sum of squared residuals. A residual is the difference between an observation’s actual value and the conditional mean assigned by the line
- The author of this book said that to achieve this goal ("minimizing SSR"), the slope's value is $\frac{cov(X,Y)}{var(X)}$ and the intercept's value is decided so that the conditional mean of response variable (usually referred to as Y) is at least right on average.
1. Slope
How does OLS use covariance to get the relationship between and ? It just takes the covariance and divides it by the variance of, i.e $\frac{cov(X,Y)}{var(X)}$. That’s it! This is roughly saying “of all the variation in X, how much of it varies along with Y?”
2. Intercept
Then, once it has its slope, it picks an intercept for the line that makes the mean of the residuals (not the squared residuals) 0, i.e., the conditional mean is at least right on average
- I can't understand why the slope explaining "of all the variation in X, how much of it varies along with Y?" helps minimizing the SSR. Plus, in the same sense, why does it help to minimize SSR, when intercepts makes the the mean of residuals 0?
