# Show $J(I-H) = 0$

This is in multiple linear regression. Given $m, n \in \mathbb{N}$ and matrices $X \in \mathbb{R}^{m \times (n+1)} (m > n + 1), H = X(X'X)^{-1}X' \in \mathbb{R}^{m\times m}$, the hat matrix, $, I = I_m$ and $J \in \mathbb{R}^{m \times m}$, a matrix of 1's, how does one show that

$J(I-H) = 0$?

I am actually trying to show

$(I-H)(H-\frac{1}{n}J) = 0$

which reduces to $J(I-H) = 0$ by noting that $HH =H$.

Apparently, this might have something to do with $J(y - X \hat{\beta}) = 0 \ \forall y \in \mathbb{R}^{1 \times (m+1)}$ where $\hat{\beta} = (X'X)^{-1}X'y$?

Is the statement true? If so why? If not, why not, and how then to prove $J(I-H) = 0$ or $(I-H)(H-\frac{1}{n}J) = 0$? I think there's some property of MLR needed.

Is there a way to do it without statistics? Like $J(I-H) = 0$ or $(I-H)(H-\frac{1}{n}J) = 0$ from just the properties of the matrices? Must one really use some properties of MLR? What are some possible counterexamples?

The "hat" matrix is a projection matrix onto the column space of $X$, $C(X)$. For any $x \in C(X)$, $Hx=x$ (see properties of projection matrix). Therefore $JH=J$ as long as you have an intercept (column of ones in your design matrix).

The matrix of all ones, $J$, can be written as $$J = \begin{pmatrix} \mathbf{1}^\intercal \\ \vdots \\\mathbf{1}^\intercal\end{pmatrix}$$ where $\mathbf{1}$ is a vector of ones. Then we have $$JH = \begin{pmatrix} \mathbf{1}^\intercal H \\ \vdots \\ \mathbf{1}^\intercal H \end{pmatrix} = \begin{pmatrix} \mathbf{1}^\intercal \\ \vdots \\ \mathbf{1}^\intercal\end{pmatrix} = J$$

if column of ones is in $\mathbf{X}$ then $H\mathbf{1} = \mathbf{1}$ and taking transpose of both sides we have $\mathbf{1}^\intercal H = \mathbf{1}^\intercal$ because $H$ is symmetric.

• There indeed is a column of ones. How exactly does it mean that JH = J? What I got so far: Column space means set of all possible linear combinations of X. It is a set of vectors. How can J be there if it is a matrix? en.wikipedia.org/wiki/Column_space#Definition
– BCLC
May 14, 2015 at 15:28
• I hope my edit answers your question. :) May 14, 2015 at 16:46
• bdeonovic, why is it that 1TH = 1T?
– BCLC
May 14, 2015 at 16:47
• I had the property on projection matrices a bit wrong, its been corrected and added another edit to answer your second comment. May 14, 2015 at 16:58
• Ugh, how do you know H1 = 1?
– BCLC
May 14, 2015 at 17:16

This can be proved with just the properties of the matrices, since the MLR is already baked into $$H$$.

If $$H$$ is defined as $$H:=X(X'X)^{-1}X'$$ then clearly $$HX=X$$. Note that $$J=11^T$$, where $$1$$ is a column of ones. Suppose that $$1$$ lives in the $$k$$th column of $$X$$, which I'll write as $$(X)_{\ast k}=1$$.

Since the $$k$$th column of $$HX$$ equals $$H$$ times the $$k$$th column of $$X$$, we can show $$H1=1$$: $$H1=H(X)_{\ast k}=(HX)_{\ast k}=(X)_{\ast k}=1$$ and therefore $$HJ=H11^T=(H1)1^T=11^T=J.$$

• forgot this already, but sounds like an answer 6 years ago me was looking for. thanks!
– BCLC
Sep 9, 2021 at 21:46
• I like this answer for using simply the property of the matrix $HX = X(X^tX)^-1(X^tX)= X$. But for $\mathbf{1}$ to be an eigenvector of $H$ it doesn't necessarily need to be a column of $X$ (it would be the easiest case to show it) and it just needs to be in the column space. Sep 25, 2021 at 18:51
• @SextusEmpiricus Indeed! If $1$ is in the column space of $X$ then there is a vector $v$ of coefficients such that $1=Xv$, and therefore $H1=H(Xv)=(HX)v=Xv=1$. I like this generalization better than the assumption that $1$ is a column in $X$. Sep 25, 2021 at 19:17

Going for the shortest answer here: just look at the first row of $$\mathbf{X}^\intercal \mathbf{H} = \mathbf{X}^\intercal$$.

• forgot this already, but sounds like an answer 6-8 years ago me was looking for. thanks!
– BCLC
Sep 25, 2021 at 20:05

$$J(H-I) = 0$$ is another way to say $$JH = J$$.

If for a vector of only ones, let's write it as $$\mathbf{1}$$, you get $$\mathbf{1}H = \mathbf{1}$$ then it also works for $$J$$ as well.

This property $$\mathbf{1}H = \mathbf{1}$$ is actually only true when the matrix $$X$$ contains the vector $$\mathbf{1}$$ in it's column space, for instance when it contains an intercept term (but an intercept is not necessary).

Counter example. Let $$X = \lbrace -1, 0, -1\rbrace$$ then $$H = \begin{bmatrix} 0.5 & 0 & -0.5 \\ 0 & 0 & 0 \\ -0.5 & 0 & 0.5 \end{bmatrix}$$

and $$\mathbf{1} H = \lbrace 0,0, 0 \rbrace \neq \mathbf{1}$$

• forgot this already, but sounds like an answer 6-8 years ago me was looking for. thanks!
– BCLC
Sep 25, 2021 at 20:05