We have a linear model \begin{align} Y=X\beta + \varepsilon \end{align} where $X$ is $n \times p$ ($n > p$) matrix of full rank. All assumptions of linear model hold.

I have to prove inequality which is connected with the elements $h_{ij}$ of the hat matrix $H=X(X'X)^{-1}X'$. The inequality is: \begin{align} 0 \le h_{ij}^{2} \le 0.25, \end{align} where $i \neq j$. Do you have any ideas how to deal with it?

  • $\begingroup$ Do you mean that for an arbitrary projection matrix this inequality holds, or you need to show it for a specific projection matrix, in which case you must know something specific about the $X$ matrix? $\endgroup$ Nov 10, 2013 at 16:14
  • 2
    $\begingroup$ Use the fact that $H=H^2$ -write out what that means. $\endgroup$ Nov 10, 2013 at 16:52
  • $\begingroup$ This question has been asked also in math SE, here : math.stackexchange.com/questions/561287/…. I have voted to close. $\endgroup$ Nov 10, 2013 at 18:38
  • 1
    $\begingroup$ Please do not cross-post; this is against SE policy. Figure out where you want this question to be & delete the other version. $\endgroup$ Nov 10, 2013 at 19:10
  • $\begingroup$ Ok, sorry for this. I have already deleted it from stackexchange. $\endgroup$
    – John Snow
    Nov 10, 2013 at 19:23

1 Answer 1


Let $H$ be an idempotent symmetric matrix. Then $H^2=H$ and for each $i=1,...,n$ we have


where we simply write the elements of the main diagonal $H^2$. We can rewrite this equality as

$$\sum_{j\neq i}h_{ij}^2=h_{ii}-h_{ii}^2 \>.$$

The quantity $h_{ii}-h_{ii}^2$ can be at most $1/4$ and we immediately get our desired result.

P.S. The inequality cannot be improved without additional assumptions, as illustrated by the following idempotent matrix

$$\begin{bmatrix}1/2 & 1/2\\1/2 & 1/2\end{bmatrix} \>.$$

  • $\begingroup$ And $h^2_{ij} \ge 0$, for $i \ne j$, follows trivially from $h_{ii}= ||\mathbf{H}\mathbf{e}_i||^2$. $\endgroup$ Nov 11, 2013 at 13:21
  • 2
    $\begingroup$ @Skull: Or more trivially...because it is the square of a real number. :-) $\endgroup$
    – cardinal
    Nov 11, 2013 at 13:23
  • $\begingroup$ ;) Sure. So remember my comment when someone happens to ask why we have $h_{ii} \ge 0$ ... $\endgroup$ Nov 11, 2013 at 13:28
  • $\begingroup$ @Skullduggery note you write $h_{ij}$ for $i\neq j$ (sic!) follows from $h_{ii}=...$. Nevertheless the fact that $h_{ii}\ge 0$ is a useful one. $\endgroup$
    – mpiktas
    Nov 11, 2013 at 13:30
  • $\begingroup$ @mpiktas: Thanks for being clear. So please enter my statement directly after the second equation of your answer and bear in mind that $h_{ii} \le 1$ follows trivially from $||\mathbf{e}_1||=1$. $\endgroup$ Nov 11, 2013 at 13:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.