# Proof of simple linear regression

Use this: \begin{align} \sum_{i=1}^{n} (X_i-\bar{X}_n) &= 0 \\ \bar{u}_{n} &=\frac{1}{n} \sum_{i=1}^{n}u_{i} \end{align} To prove this: $$\sum_{i=1}^{n} (X_i-\bar{X}_n)(\beta_i(X_i-\bar{X}_i)+u_i-\bar{u}_n)=\beta\sum_{i=1}^{n} (X_i-\bar{X}_n)^{2}+\sum_{i=1}^{n}(X_i-\bar{X}_n)u_i$$

However, instead I get: $$\beta\sum_{i=1}^{n} (X_i-\bar{X}_n)^{2}+\sum_{i=1}^{n}(X_i-\bar{X}_n)u_i+\sum_{i=1}^{n}(X_i-\bar{X}_n)\bar{u}_n$$

Why doesn't $\sum_{i=1}^{n}(X_i-\bar{X}_n)u_i=0$, but $\sum_{i=1}^{n}(X_i-\bar{X}_n)\bar{u}_n=0$ does? And how do I make $\sum_{i=1}^{n}(X_i-\bar{X}_n)\bar{u}_n=0$ disappear?

• Hint: think about why we may pull $\bar{u}_n$ before the summation sign, but not $u_i$. Commented Nov 30, 2016 at 11:55
• What is $u$? Please define the terms and symbols you use. Commented Nov 30, 2016 at 14:29
• @MatthewDrury I have added it to the main text now. Commented Dec 1, 2016 at 13:44
• @ChristophHanck Is it because $\bar{u}_{n}$ is a constant, while $u_{i}$ is not? Commented Dec 1, 2016 at 13:46
• Yes, that is right - for $\bar{u}_n$ the average has already been taken over $i$, so it is constant with respect to $i$. Commented Dec 1, 2016 at 13:47

$\overline{u}_n$ is a constant not depending on $i$. Hence you can write $\sum_{i=1}^n (X_i - \overline{X}_n)\overline{u}_n = \overline{u}_n (\sum_{i=1}^n (X_i - \overline{X}_n))$ and use that $\sum_{i=1}^n (X_i - \overline{X}_n) = 0$ which gives you $\overline{u}_n * 0 =0$

To answer the questions you were asking in the comments, I'll try to explain in a very detailed way, but most of it was already said in the comments somehow. $$\sum_{i=1}^n(X_i-\overline{X}_n)\overline{u}_n= \sum_{i=1}^nX_i\overline{u}_n - \sum_{i=1}^n\overline{X}_n\overline{u}_n=\overline{u}_n\sum_{i=1}^nX_i - \overline{u}_n\sum_{i=1}^n\overline{X}_n=\\ \overline{u}_n\sum_{i=1}^nX_i - \overline{u}_nn\overline{X}_n=\overline{u}_nn\overline{X}_n - \overline{u}_nn\overline{X}_n=0$$ The first equalitiy is just multiplying out. The second equalitiy holds because $\overline{u}_n$ is a constant (not depending on $i$). The third equalitiy holds becaus $\overline{X}_n$ also does not depend on $i$, hence it is just summed $n$ times. The fourth equality holds because the sum of the $X_i$'s is just $n$ times their mean. The fifths is clear. Now let's consider the other sum. $$\sum_{i=1}^n(X_i-\overline{X}_n)u_i=\sum_{i=1}^nX_iu_i-\sum_{i=1}^n\overline{X}_nu_i = \sum_{i=1}^nX_iu_i-\overline{X}_{n}\sum_{i=1}^nu_i = \\ \sum_{i=1}^nX_iu_i-\overline{X}_{n}n\overline{u}_n = n\overline{(Xu)_n}-n\overline{X}_{n}\overline{u}_n \neq 0 \quad \text{(in general)}$$ The first equality is just multiplying out. The second hold because $\overline{X}_n$ does not depend on $i$. The third equality holds because the sum of the $u_i$'s is just $n$ times their mean. The fourth equation is rewriting the sum of the $X_i$ times $u_i$, as $n$ times their mean $\overline{(Xu)_n} := (X_1u_1+X_2u_2 + \ldots +X_nu_n)/n$ and since $\overline{(Xu)_n} \neq \overline{X}_{n}\overline{u}_n$ (e.g. $\frac{1*4 + 2*5 + 3*6}{3} = 10.666 \neq 10 =\frac{1+2+3}{3} * \frac{4+5+6}{3}$) the last inequality follows.

• Thank you, @Schlaftablette! That is a great explanation of my problem. Commented Dec 3, 2016 at 11:40
• However, I am wondering something about $\overline{u}_n\sum_{i=1}^nX_i$, why is $\overline{u}_n\sum_{i=1}^nX_i=n\bar{Xu}_{n}$ true? Isn't the definition of $\bar{X}_{n}=\frac{1}{n}\sum_{i=1}^nX_i$? or wouldn't $\sum_{i=1}^nu_{i}=nu_{i}$? Commented Dec 3, 2016 at 11:56