# Linear regression $y_i=\beta_0 + \beta_1x_i + \epsilon_i$ covariance between $\bar{y}$ and $\hat{\beta}_1$

I am currently reading through slides from Georgia Tech on linear regression and came across a section that has confused me. It states for $$y_i=\beta_0+\beta_1x_i+\epsilon_i$$ where $$\epsilon_i \sim N(0,\sigma^2)$$ and $$\hat{\beta}_1=\frac{\sum_{i=1}^n(x_i-\bar{x})y_i}{\sum_{i=1}^n(x_i-\bar{x})^2}$$ the covariance is \begin{align*} Cov(\bar{y},\hat{\beta}_1)&=\frac{1}{\sum_{i=1}^n(x_i-\bar{x})^2}Cov\Big(\bar{y},\sum_{i=1}^n(x_i-\bar{x})y_i\Big) \\ &=\frac{\sum_{i=1}^n(x_i-\bar{x})}{\sum_{i=1}^n(x_i-\bar{x})^2}Cov\Big(\bar{y},y_i\Big) \\ &=\frac{\sum_{i=1}^n(x_i-\bar{x})}{\sum_{i=1}^n(x_i-\bar{x})^2}\frac{\sigma^2}{n} \\ &= 0 \end{align*} Now, I assume that it becomes 0 from the $$\sum_{i=1}^n(x_i-\bar{x})=0$$ term. However, what is confusing me is how we can pull out the $$\sum_{i=1}^n(x_i-\bar{x})$$ term from the $$Cov\Big(\bar{y},\sum_{i=1}^n(x_i-\bar{x})y_i\Big)$$ since the $$y_i$$ is part of the summation and not constant for all $$i$$ values (or so I thought).

You don't pull out $$\sum_{i=1}^n (x_i - \bar{x})$$. Instead, you pull out $$(x_i - \bar{x})$$ $$n$$ times. Throughout, the $$(x_i)_{1 \leq i \leq n}$$ sequence is taken to be non-random. The way the original argument is written up is a little confusing. The following may be easier to understand: \begin{align*} Cov(\bar{y},\hat{\beta}_1)&=\frac{1}{\sum_{j=1}^n(x_j-\bar{x})^2}Cov\Big(\bar{y},\sum_{i=1}^n(x_i-\bar{x})y_i\Big) \\ &=\frac{1}{\sum_{j=1}^n(x_j-\bar{x})^2}Cov\Big(\bar{y},(x_1-\bar{x})y_1 + \dotsm + (x_n-\bar{x})y_n \Big) \\ &=\frac{(x_1-\bar{x})Cov(\bar{y}, y_1) + \dotsm + (x_n-\bar{x})Cov(\bar{y}, y_n)}{\sum_{j=1}^n(x_j-\bar{x})^2} \\ &=\frac{\sum_{i=1}^n(x_i-\bar{x})Cov\Big(\bar{y},y_i\Big)}{\sum_{j=1}^n(x_j-\bar{x})^2} \\ &=\frac{\sum_{i=1}^n(x_i-\bar{x})}{\sum_{j=1}^n(x_j-\bar{x})^2}\frac{\sigma^2}{n} \\ &= 0. \end{align*} To evaluate $$Cov(\bar{y}, y_i)$$ note that $$Cov(y_j, y_i)=0$$ for $$j \neq i$$ and write \begin{align*} Cov(\bar{y}, y_i) =& n^{-1} Cov(y_1 + \dotsm + y_n, y_i) \\ =& n^{-1} \Big(0 + \dotsm + 0 + Cov(y_i, y_i) + 0 \dotsm + 0\Big) \\ =& n^{-1} Var(y_i) = n^{-1} Var(\beta_0 + \beta_1 x_i + \epsilon_i) = n^{-1} Var(\epsilon_i) = \sigma^2/n. \end{align*}
• @AndreasDzemski So we treat $(x_i-\bar{x})$ as a constant? If that's the case, then why don't we still keep a $\sum_{i=1}^ny_i$ in the $Cov$ function as well? Making it $Cov(\bar{y},\sum_{i=1}^ny_i)$ – strwars Oct 15 '19 at 20:07
• Yes the $x_i$ variables are treated as non-random. $Cov(\bar y \sum_i (x_i -\bar x)y_i) = Cov(\bar y, (x_1 - \bar x)y_1 + ... + (x_N - \bar x)y_N) = (x_1-\bar x)Cov(\bar y,y_1) + ... + (x_N - \bar x) Cov(\bar y,y_N) = \sum_{i}( x_i - \bar x) Cov(\bar y,y_i) = \sum_{i}( x_i - \bar x) Cov(\bar \epsilon,\epsilon_i ) = (1/n) \sum_{i}( x_i - \bar x) Cov(\epsilon_i,\epsilon_i)$ – Jesper for President Oct 15 '19 at 20:17
• @JesperHybel thank you for the explanation, it has cleared a lot up! I assume the $Cov(\bar{\epsilon_i},\epsilon_i)$ is converted by $Cov(\bar{\epsilon_i},\epsilon_i)=Cov\Big(\frac{1}{n}(\epsilon_1+\epsilon_2+...+\epsilon_n),\epsilon_i\Big)=\frac{1}{n}Cov(\sum_{i=1}^n\epsilon_i,\epsilon_i)$ and $\sum_{i=1}^n(x_i+\bar{x})=0$ makes the whole thing 0. – strwars Oct 15 '19 at 22:43