2
$\begingroup$

I'm trying to finish proving that in simple-regression analysis, $h_i = \frac{1}{n} + \frac{(X_i - \bar{X})^2}{\sum_{j=1}^{n}(X_j - \bar{X})^2}$, where $h_i := h_{ii} = \sum_{j=1}^nh_{ij}^2$, the entries of the hat matrix $H$. I've done the following so far: for $x_i' = (1, X_i)$, $$x_i'(X'X)^{-1}x_i = \begin{bmatrix} 1 & X_i \end{bmatrix}\left(\begin{bmatrix} n & \sum X_i\\ \sum X_i & \sum X_i^2 \end{bmatrix}\right)^{-1}\begin{bmatrix} 1\\ X_i\end{bmatrix} = \begin{bmatrix} 1 & X_i \end{bmatrix}\frac{1}{n\sum X_i^2 - (\sum X_i)^2}\begin{bmatrix} \sum X_i^2 & -\sum X_i\\ -\sum X_i & n \end{bmatrix}\begin{bmatrix} 1\\ X_i\end{bmatrix} = \frac{1}{n\sum X_i^2 - (\sum X_i)^2}\begin{bmatrix}0 & nX_i- \sum X_i^2 \end{bmatrix}\begin{bmatrix} 1\\ X_i\end{bmatrix} = \frac{nX_i^2 - \sum X_i^2}{n\sum X_i^2 - (\sum X_i)^2}.$$ Now, I'm stuck... I do understand that $\frac{1}{n} \leq h_i \leq 1$, but am not sure where else to go from here. Thanks for any help!

$\endgroup$
1

1 Answer 1

3
$\begingroup$

You have a misuse of indices. Inside the matrix, it should be $j$ because the summation index is different from $i$ of $x_i$ at the beginning. If we continue with $j$'s: $$\begin{align}h_{ii}&=\frac{\sum_j X_j^2+nX_i^2-2\sum_j X_j}{n\sum_jX_j^2-(\sum_jX_j)^2}=\frac{\sum_j X_j^2+nX_i^2-2n\bar{X}+n\bar{X}^2-n\bar{X}^2}{n\sum_jX_j^2-n^2\bar{X}^2}\\&=\frac{\sum_jX_j^2-n\bar{X}^2}{n\sum_jX_j^2-n^2\bar{X}^2}+\frac{(X_i-\bar{X})^2}{\sum_jX_j^2-n\bar{X}^2}\\&=\frac{1}{n}+\frac{(X_i-\bar{X})^2}{\sum_j(X_j-\bar{X})^2}\end{align}$$

$\endgroup$
4
  • $\begingroup$ When I replace the $X_i$ with $X_j$ in the $(X'X)^{-1}$ matrix and do the multiplication, I end up getting $\sum_j X_j^2+nX_i^2-2X_i\sum_j X_j$ in the numerator for $h_{ii}$... am I understanding you correctly? Also, as for your first step above, I see that you add and subtract $n\bar{X}^2$ in the numerator, but what exactly are you doing to the denominator? Thanks. $\endgroup$
    – Jake
    Oct 20, 2019 at 20:49
  • $\begingroup$ Yes, you should be getting that. For the denominator, I set $\sum_j X_j=n\bar{X}$, which is also done in the numerator. $\endgroup$
    – gunes
    Oct 20, 2019 at 20:50
  • $\begingroup$ I just have one more question--how do you simplify $\sum_jX_j^2-n\bar{X}^2$ to be $\sum_j(X_j-\bar{X})^2$? I've been grappling with the algebra for a while and I can't get it to work. $\endgroup$
    – Jake
    Oct 20, 2019 at 22:20
  • $\begingroup$ @Jake Open up the latter, and put $\sum_j X_j = n\bar{X}$. $\endgroup$
    – gunes
    Oct 21, 2019 at 5:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.