1
$\begingroup$

I am considering a linear regression model $Y_i = X_i^T \beta + \epsilon_i, i = 1,2,\dots,n$. where $X_i \in \mathbb{R}^p$. $\epsilon_i$'s are independent copies of random error $\epsilon \in \mathbb{R}^1$ with mean 0 and variance $\sigma^2$. We have the hat matrix given by $$ H = X^T (X^TX)^{-1} X$$ I am trying to show $h_{ij}^2 \leq h_{ii} h_{kk}$. I know that $$h_{ij} = X_i^T (X^T X)^{-1} X_j$$ and that $H^2 = H$ because it is idempotent. This gives me $\sum_{k=1}^n h_{ik}^2 = h_{ii}$ but I don't know how to prove $h_{ij}^2 \leq h_{ii} h_{jj}$.

$\endgroup$
  • $\begingroup$ H is the covariance matrix of the parameters. That is the covariance divided by (sqrt) variance of elements is the correlation that is always less than one. $\endgroup$ – TPArrow Aug 12 '16 at 8:49
  • $\begingroup$ H is the covariance matrix of fitted values..$ Cov(\hat{Y}) = H\sigma^2$. Now your argument works. Thanks! $\endgroup$ – user111092 Aug 12 '16 at 12:01
2
$\begingroup$

Your friend is the Cauchy-Schwarz inequality. Let $H=X (X^T X)^{-1} X^T$ where $X$ is the design matrix in the linear model $Y_i = X_i^T \beta + \epsilon_i$, $X_i$ is the $i$th row of the design matrix. Since $(X^TX)^{-1}$ is positive definite, we can define an inner product by $$ h_{ij} =\langle X_i, X_j \rangle = X_i^T (X^TX)^{-1} X_j $$ and by the Cauchy-Schwarts inequality $$ h_{ij} = |h_{ij}| = |\langle X_i,X_j\rangle |\le \sqrt{\langle X_i,X_i\rangle \langle X_j, X_j \rangle} =\sqrt{h_{ii} h_{jj}} $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.