1
$\begingroup$

I am considering a linear regression model $Y_i = X_i^T \beta + \epsilon_i, i = 1,2,\dots,n$. where $X_i \in \mathbb{R}^p$. $\epsilon_i$'s are independent copies of random error $\epsilon \in \mathbb{R}^1$ with mean 0 and variance $\sigma^2$. We have the hat matrix given by $$ H = X^T (X^TX)^{-1} X$$ I am trying to show $h_{ij}^2 \leq h_{ii} h_{kk}$. I know that $$h_{ij} = X_i^T (X^T X)^{-1} X_j$$ and that $H^2 = H$ because it is idempotent. This gives me $\sum_{k=1}^n h_{ik}^2 = h_{ii}$ but I don't know how to prove $h_{ij}^2 \leq h_{ii} h_{jj}$.

$\endgroup$
2
  • $\begingroup$ H is the covariance matrix of the parameters. That is the covariance divided by (sqrt) variance of elements is the correlation that is always less than one. $\endgroup$
    – TPArrow
    Aug 12, 2016 at 8:49
  • $\begingroup$ H is the covariance matrix of fitted values..$ Cov(\hat{Y}) = H\sigma^2$. Now your argument works. Thanks! $\endgroup$
    – user111092
    Aug 12, 2016 at 12:01

1 Answer 1

2
$\begingroup$

Your friend is the Cauchy-Schwarz inequality. Let $H=X (X^T X)^{-1} X^T$ where $X$ is the design matrix in the linear model $Y_i = X_i^T \beta + \epsilon_i$, $X_i$ is the $i$th row of the design matrix. Since $(X^TX)^{-1}$ is positive definite, we can define an inner product by $$ h_{ij} =\langle X_i, X_j \rangle = X_i^T (X^TX)^{-1} X_j $$ and by the Cauchy-Schwarts inequality $$ h_{ij} = |h_{ij}| = |\langle X_i,X_j\rangle |\le \sqrt{\langle X_i,X_i\rangle \langle X_j, X_j \rangle} =\sqrt{h_{ii} h_{jj}} $$

$\endgroup$
1
  • $\begingroup$ should the first equality in the last display not be $\leq$? $\endgroup$ Jul 13, 2022 at 15:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.