• What would be the intuitive explanation for the slope of Ordinary Least Squares(OLS), which is $\frac{cov(X,Y)}{var(X)}$ contributes minimizing the sum of squared residuals?

  • In the same sense, why the intercept is decided to make the mean of the residuals 0 (not the mean of squared residuals, but just the mean of residuals) ? In other words, conditional mean of a response varialbe is at least right on average. What would be the intuitive explanation for this should be the case?

More details

OLS picks the line that gives the lowest sum of squared residuals. A residual is the difference between an observation’s actual value and the conditional mean assigned by the line

  • The author of this book said that to achieve this goal ("minimizing SSR"), the slope's value is $\frac{cov(X,Y)}{var(X)}$ and the intercept's value is decided so that the conditional mean of response variable (usually referred to as Y) is at least right on average.

1. Slope

How does OLS use covariance to get the relationship between and ? It just takes the covariance and divides it by the variance of, i.e $\frac{cov(X,Y)}{var(X)}$. That’s it! This is roughly saying “of all the variation in X, how much of it varies along with Y?”

2. Intercept

Then, once it has its slope, it picks an intercept for the line that makes the mean of the residuals (not the squared residuals) 0, i.e., the conditional mean is at least right on average

  • I can't understand why the slope explaining "of all the variation in X, how much of it varies along with Y?" helps minimizing the SSR. Plus, in the same sense, why does it help to minimize SSR, when intercepts makes the the mean of residuals 0?
  • $\begingroup$ Geometric interpretations make these relationships clearer, even obvious. See my posts at stats.stackexchange.com/a/164644/919, stats.stackexchange.com/a/54970/919, stats.stackexchange.com/a/66295/919, and stats.stackexchange.com/a/513296/919 for detailed explanations from various perspectives and varying levels of mathematical background. $\endgroup$
    – whuber
    Feb 15 at 14:18
  • $\begingroup$ @whuber Thanks for the comment, but none of the replies mentioned in your comment are more of a math heavy answers, so not exactly what I am looking for. $\endgroup$
    – Eiffelbear
    Feb 15 at 16:16
  • $\begingroup$ @whuber Plus, none of the answers directly respond to my questions... $\endgroup$
    – Eiffelbear
    Feb 15 at 16:26
  • 1
    $\begingroup$ Please, then, clarify your questions. Many, many derivations of these formulas are provided on this site. $\endgroup$
    – whuber
    Feb 15 at 18:33