1
$\begingroup$

How did they go from (1) to (2): \begin{align*} S_{xx} &= \sum(X_i - \bar{X})^2 \tag1 \\ &= \sum(X_i - \bar{X}) X_i \tag2 \\ &= \sum X_i^2 - \left(\sum{X_i}\right)^2/n \\ &= \sum X_i^2 - n \bar{X} \end{align*} In (2), are they simply saying that $(X_i - \bar{X}) = X_i$? Why is that so?

It is also seen here in OLS equation: $$b_1 = \frac{\sum X_i Y_i - \left[\left(\sum X_i \right) \left(\sum Y_i \right)\right]/n}{\sum X_i^2 - \left( \sum X_i\right)^2 /n} = \frac{ \sum\left(X_i -\bar{X}\right) \left(Y_i - \bar{Y}\right)}{\sum \left(X_i - \bar{X} \right)^2}$$

The technique is used again in the denominator, when they go from middle equation to the right. Why is it?

Improve this question
$\endgroup$
  • $\begingroup$ Essentially it's just expanding the quadratic, simplifying, collecting & cancelling terms. See here for example: stats.stackexchange.com/questions/256179/… -- the answer covers essentially this in the middle (apart from some minor simplifications to match the derivation here). [I'm debating whether this is different enough in detail to stand on its own rather than close as a duplicate.] $\endgroup$ – Glen_b May 8 '17 at 15:01
  • 1
    $\begingroup$ As everyone is implying and saying, it's a simple algebraic consequence of the definition of $\bar{X} = \frac{1}{n} \sum_{i=1}^n X_i$ $\endgroup$ – Matthew Gunn May 8 '17 at 15:02
  • $\begingroup$ @Glen Because this exact question (a) has appeared several dozen times but (b) is almost impossible to search for, I think it's well worth while closing the duplicates and pointing them at some good, canonical answers: we can hope that will help in future searches. $\endgroup$ – whuber♦ May 8 '17 at 15:07
  • $\begingroup$ I just changed the title there a few minutes ago in the hope of making it easier to find. $\endgroup$ – Glen_b May 8 '17 at 15:12
  • $\begingroup$ No No No, Scott's answer is MUCH better! Simple and elegant! The other question does not answer my question, albeit similar, they never shown the identity at the end, which is the most important thing! $\endgroup$ – user13985 May 8 '17 at 15:44
3
$\begingroup$

$$ \sum (X_i - \bar{X})^2 $$ $$ \sum(X_i - \bar{X})(X_i - \bar{X}) $$ $$ \sum (X_i^2 - 2\bar{X}X_i + \bar{X}^2) $$ $$ \sum \left[(X_i^2 - \bar{X}X_i) + (\bar{X}^2 - \bar{X}X_i)\right] $$ $$ \sum \left[(X_i - \bar{X})X_i + (\bar{X} - X_i)\bar{X}\right] $$

We can "distribute" the $\Sigma$ over those two summands. The second one turns out to be zero

$$ \sum (\bar{X} - X_i)\bar{X} $$ $$ \bar{X} \sum (\bar{X} - X_i) $$ $$ \bar{X} (\sum \bar{X} - \sum X_i) $$ $$ \bar{X} (n\bar{X} - n\bar{X}) $$ $$ 0 $$

Improve this answer
$\endgroup$
  • $\begingroup$ Is this correct? $$\sum X_i^2 - 2 \sum X_i \bar{X} + \sum \bar{X}^2 = \sum \left[X_i^2 - 2 X_i \bar{X} + \bar{X}^2 \right]$$ I think that's the part that got me stuck! $\endgroup$ – user13985 May 8 '17 at 16:29
  • $\begingroup$ Between steps 2 and 3 is just the distributive property (FOIL, recall from algebra). Between steps 3 and 4 is: Take those 2 $\bar{X}X_i$ that you have, and give one to the first term and one to the second term. The parenthesis are just there to accentuate the grouping. $\endgroup$ – Him May 8 '17 at 19:53
  • $\begingroup$ .... The equality in your comment is correct, yes, but you shouldn't need that specific result in the proof. $\endgroup$ – Him May 8 '17 at 19:54
  • $\begingroup$ Can you explain why $$ \sum(X_i - \bar{X})(X_i - \bar{X}) = \sum(X_i - \bar{X}) X_i $$ What is the intuition behind replacing $(X_i - \bar{X})$ with $X_i$ but still have the same result? $\endgroup$ – user13985 May 8 '17 at 20:01
1
$\begingroup$

Simple numerical example

Let $X_1 = 1$, $X_2 = 3$, $X_3 = 8$

Then $\bar{X} = \frac{1}{3} \left( 1 + 3 + 8\right)$ = 4

It is not at all correct to say $(X_1 - \bar{X}) = X_1$ which would be equivalent to saying that (1 - 4) = 1

The point is that

$$ \sum_i \bar{X} \left( X_i - \bar{X} \right) = 0$$

because $\sum_i X_i = n \bar{X}$. In this example $1 + 3 + 8 = 3 \cdot 4 = 12$

In this example, the statement $\sum_i \bar{X} \left( X_i - \bar{X} \right) = 0$ would be:

$$4\left( 1 - 4 \right) + 4 \left( 3 - 4 \right) + 4 \left(8 - 4\right) = 0$$

If you factor our $\bar{X}$:

$$ \bar{X} \left[ \sum_i \left( X_i - \bar{X} \right) \right] = 4 \left[ ( 1 - 4) + (3 - 4) + (8 - 4) \right] = 0$$

Improve this answer
$\endgroup$
  • $\begingroup$ Yes, this makes sense. $(X_i - \bar{X})$ is essentially residuals around the mean, which sums to zero, and $\bar{X}$ is a constant. But, I was asking why this $$\sum (X_i - \bar{X})^2 $$ results in this $$\sum (X_i - \bar{X})X_i $$ in its interpretation. I have understood how the math works out. $\endgroup$ – user13985 May 8 '17 at 19:24
  • $\begingroup$ @user13985 This is the ubiquitous identity that $\operatorname{Var}(X) = \operatorname{E}[X^2] - \operatorname{E}[X]^2$. See this question $\endgroup$ – Matthew Gunn May 8 '17 at 19:35
  • $\begingroup$ I even wrote a short R code: $$\text{x <- c(1, 2, 4, 7, 11) }\\ \text{xbar <- mean(x)} \\ \text{(x - xbar) * x} \\ \text{[1] -4 -6 -4 14 66 }\\ \text{(x - xbar) * (x - xbar) }\\ \text{[1] 16 9 1 4 36 }\\ \text{sum((x - xbar) * x) }\\ \text{[1] 66 }\\ \text{sum((x-xbar) * (x - xbar)) }\\ \text{[1] 66 }$$ But, it still don't know why it works. I feel like there must be an interpretation to this. $\endgroup$ – user13985 May 8 '17 at 19:44

Not the answer you're looking for? Browse other questions tagged or ask your own question.