# Linear regression question on Idempotent matrix and leverage points

I am considering a linear regression model $Y_i = X_i^T \beta + \epsilon_i, i = 1,2,\dots,n$. where $X_i \in \mathbb{R}^p$. $\epsilon_i$'s are independent copies of random error $\epsilon \in \mathbb{R}^1$ with mean 0 and variance $\sigma^2$. We have the hat matrix given by $$H = X^T (X^TX)^{-1} X$$ I am trying to show $h_{ij}^2 \leq h_{ii} h_{kk}$. I know that $$h_{ij} = X_i^T (X^T X)^{-1} X_j$$ and that $H^2 = H$ because it is idempotent. This gives me $\sum_{k=1}^n h_{ik}^2 = h_{ii}$ but I don't know how to prove $h_{ij}^2 \leq h_{ii} h_{jj}$.

• H is the covariance matrix of the parameters. That is the covariance divided by (sqrt) variance of elements is the correlation that is always less than one. – TPArrow Aug 12 '16 at 8:49
• H is the covariance matrix of fitted values..$Cov(\hat{Y}) = H\sigma^2$. Now your argument works. Thanks! – user111092 Aug 12 '16 at 12:01

Your friend is the Cauchy-Schwarz inequality. Let $$H=X (X^T X)^{-1} X^T$$ where $$X$$ is the design matrix in the linear model $$Y_i = X_i^T \beta + \epsilon_i$$, $$X_i$$ is the $$i$$th row of the design matrix. Since $$(X^TX)^{-1}$$ is positive definite, we can define an inner product by $$h_{ij} =\langle X_i, X_j \rangle = X_i^T (X^TX)^{-1} X_j$$ and by the Cauchy-Schwarts inequality $$h_{ij} = |h_{ij}| = |\langle X_i,X_j\rangle |\le \sqrt{\langle X_i,X_i\rangle \langle X_j, X_j \rangle} =\sqrt{h_{ii} h_{jj}}$$