Why is a regression coefficient covariance/variance Title. I'd like to understand the intuition behind how regression coefficients are calculated, and why $$\frac{cov(x,y)}{var(x)}$$ gives a regression coefficient for dependent variable y and independent variable x? This feels like an elementary question but I have looked around the internet and can't anything answering this question specifically.
 A: 
Why $$\frac{cov(x,y)}{var(x)}$$ gives a regression coefficient for dependent variable y and independent variable x?

A linear regression coefficient tells us: If predictor variable $x$ increases by 1, what is the expected increase in outcome variable $y$?
The answer to this question depends in large part on the scales on which $x$ and $y$ are measured. E.g., if $x$ is a measure of length, imagine measuring in millimeters or centimeters; the variance of measurements in millimeters will be $10^2$ times the variance of the same measurements in centimeters; the covariance will be multiplied by 10. Note, $cov(x,y)$ is determined by three things:

*

*the linear association between $x$ and $y$;

*the scale of $x$;

*the scale of $y$.

Because of 2) and 3), I would call the covariance an unstandardized measure of association. Its value is difficult to interpret, because what would be a large and what would be a small value depends on the scales of $x$ and $y$. The correlation coefficient, however, gives us a standardized measure of association: It is 'corrected' for the scales on which $x$ and $y$ are measured:
$$cor(x,y)=\frac{cov(x,y)}{\sqrt{var(x) \cdot var(y)}}$$
The correlation coefficient tells us: If $x$ increases by $\sqrt{var(x)}$, how many $\sqrt{var(y)}$s will outcome $y$ increase? Thus, with a correlation coefficient of 1, an increase of 1 SD in $x$ is associated with an increase of 1 SD in $y$.
Now, the regression coefficient quantifies the expected increase in $y$, when $x$ increases by 1. We thus need to 'correct' the covariance between $x$ and $y$ for the scale of $x$. We can do that by simply dividing:
$$\frac{cov(x,y)}{var(x)}$$
Note that if we would 'reverse' the problem, and ask the question: If $y$ increases by 1, what is the expected increase in $x$? We can compute the answer as follows:
$$\frac{cov(x,y)}{var(y)}$$
A: In simple linear regression we are dealing with the model
\begin{eqnarray*}
y_i = \beta_0 + \beta_1 x_i + \varepsilon_i, \quad \mbox{for} \quad i=1,\cdots,n.
\end{eqnarray*}
Let $\boldsymbol{y} = \left(y_1, \cdots, y_n\right)^{\prime}$, $\boldsymbol{x} = \left(x_1, \cdots, x_n\right)^{\prime}$, and $\boldsymbol{\beta} = \left(\beta_0, \beta_1\right)$.  Moreover, the design matrix is $\boldsymbol{X} = \left[\boldsymbol{1}_n, \boldsymbol{x} \right]$.  It may be shown that the MLE of $\boldsymbol{\beta}$ is
\begin{eqnarray*}
\widehat{\boldsymbol{\beta}} &=& \left(\boldsymbol{X}^{\prime}\boldsymbol{X}\right)^{-1} \boldsymbol{X}^{\prime} \boldsymbol{y} \\
&=& \frac{1}{n\sum_{i=1}^n x_i^2 - n^2 \bar{x}^2} \begin{pmatrix}
\sum_{i=1}^n x_i^2 & -n\bar{x} \\  
-n\bar{x} & n 
\end{pmatrix}
\begin{pmatrix}
n \bar{y} \\
\sum_{i=1}^n x_iy_i
\end{pmatrix}.
\end{eqnarray*}
Clearly, the second term, $\widehat{\beta}_1$, is given by
\begin{eqnarray*}
\widehat{\beta}_1 &=& \frac{n\sum_{i=1}^n x_iy_i - n^2 \bar{x}\bar{y}}{n\sum_{i=1}^n x_i^2 - n^2 \bar{x}^2} \\
&=& \frac{\frac{\sum_{i=1}^n x_iy_i - n \bar{x}\bar{y}}{n-1}}{\frac{\sum_{i=1}^n x_i^2 - n \bar{x}^2}{n-1}} \\
&=& \frac{cov(x,y)}{var(x)},
\end{eqnarray*}
where $var$ and $cov$ represent their sample counterparts.
A: The first thing to mention is that the $\frac{cov(x,y)}{var(x)}$ will only be the $b_1$ in the case of this regression: $y = b_0 + b_1x$, that is, when we only have two variables. If we have a third, for example, this breaks down as you can test with the R code below:
set.seed(2021)
N <- 1000
X <- rnorm(N)
Y <- X + rnorm(N)
lm(Y~X)
cov(X,Y)/var(X)

Running the code above you will get the same value from the last two lines, which is $0.9882329$. But if you run the code below:
set.seed(2021)
N <- 1000
X <- rnorm(N)
Y <- X + rnorm(N)
Z <- X + rnorm(N)
lm(Y~X + Z)
cov(X,Y)/var(X)

You will get $0.9887337$ for $b_1$ and $0.9882329$ for $\frac{cov(X,Y)}{var(X)}$. And even though these values are close here, this is not always the case. If I had Z <- Y + rnorm(N), you would see $0.47370$ for $b_1$.
Now for your question. The variance measures how spread are the data points of a variable when compared to its mean. The covariance, in a way, measures if the spread in variable X follows the spread in variable Y. See the example below.
$cov(X,Y) = E[(X-E[X])(Y-E[Y])]$
Let's look at a data point at a time. We estimated that the mean for $X$ is $5$ and for $Y$ it is $10$. The first value we have for $X$ in the dataset is $20$, which is above its mean ($20\gt 5$). The first value we have for $Y$ is $0$, which is below its mean ($0 \lt 10$). Let's plug these values in the equation for covariance.
$(X-E[X])(Y-E[Y]) = (20-5)(0-10) = (15)(-10) = -150$.
Another data point now. $6$ for $X$ and $5$ for $Y$. Again, we will get a negative value.
$(X-E[X])(Y-E[Y]) = (6-5)(5-10) = (1)(-5) = -5$
If we keep doing this, and keep seeing this behaviour, it's easy to see that at the end we will have a negative covariance. $X$ values will to some extent be frequently above its mean, while $Y$ values will to some extent be frequently below its mean. Knowing that the Pearson correlation coefficient (PCC) is the standardised covariance ($ r = \frac{cov(X,Y)}{\sigma X \sigma Y}$), we would then equally get a negative value, that is, a negative correlation.
What does a negative correlation tell you right away? If I don't tell you the value of the PCC, the only thing the sign gives you is the "orientation" of the slope (increasing or decreasing). What's the $b_1$ coefficient in $y = b_0 + b_1x$? The slope! :-)
Based on that, I assume it's clear that the covariance of $X$ and $Y$ is proportional to the slope ($b_1$) of the regression line of $Y$, as dependent variable, and $X$ as independent variable.
However, there is a catch here. If you try to do a regression swapping who is the independent variable and who is the dependent one, you will realize the coefficients change! See the code below.
set.seed(2021)
N <- 1000
X <- rnorm(N)
Y <- X + rnorm(N)
lm(Y~X)$coefficients
lm(X~Y)$coefficients

$b_1$ for the first regression will be $0.98823293$, as we already saw, but for the second regression we will get $0.50075232$. Based on what I said about $cov(X,Y)$, someone may think of trying $cov(Y,X)$ but covariance is symmetric, that is, $cov(X,Y) = cov(Y,X)$. If you divide this by the variance of the independent variable, you will get the right value. The code below shows this:
set.seed(2021)
N <- 1000
X <- rnorm(N)
Y <- X + rnorm(N)
lm(Y~X)$coefficients
lm(X~Y)$coefficients
cov(Y,X)
cov(X,Y)

cov(X,Y)/var(X)
cov(X,Y)/var(Y)

For the derivation of the least squares estimators of the slope and intercept in linear regression, this is a complete and slow-paced video.
