2
$\begingroup$

This is in multiple linear regression. Given $m, n \in \mathbb{N}$ and matrices $X \in \mathbb{R}^{m \times (n+1)} (m > n + 1), H = X(X'X)^{-1}X' \in \mathbb{R}^{m\times m}$, the hat matrix, $, I = I_m$ and $J \in \mathbb{R}^{m \times m}$, a matrix of 1's, how does one show that

$J(I-H) = 0$?

I am actually trying to show

$(I-H)(H-\frac{1}{n}J) = 0$

which reduces to $J(I-H) = 0$ by noting that $HH =H$.

Apparently, this might have something to do with $J(y - X \hat{\beta}) = 0 \ \forall y \in \mathbb{R}^{1 \times (m+1)}$ where $\hat{\beta} = (X'X)^{-1}X'y $?

Is the statement true? If so why? If not, why not, and how then to prove $J(I-H) = 0$ or $(I-H)(H-\frac{1}{n}J) = 0$? I think there's some property of MLR needed.

Is there a way to do it without statistics? Like $J(I-H) = 0$ or $(I-H)(H-\frac{1}{n}J) = 0$ from just the properties of the matrices? Must one really use some properties of MLR? What are some possible counterexamples?

$\endgroup$
2
$\begingroup$

The "hat" matrix is a projection matrix onto the column space of $X$, $C(X)$. For any $x \in C(X)$, $Hx=x$ (see properties of projection matrix). Therefore $JH=J$ as long as you have an intercept (column of ones in your design matrix).

EDIT to address comment:

The matrix of all ones, $J$, can be written as $$J = \begin{pmatrix} \mathbf{1}^\intercal \\ \vdots \\\mathbf{1}^\intercal\end{pmatrix}$$ where $\mathbf{1}$ is a vector of ones. Then we have $$JH = \begin{pmatrix} \mathbf{1}^\intercal H \\ \vdots \\ \mathbf{1}^\intercal H \end{pmatrix} = \begin{pmatrix} \mathbf{1}^\intercal \\ \vdots \\ \mathbf{1}^\intercal\end{pmatrix} = J$$

if column of ones is in $\mathbf{X}$ then $H\mathbf{1} = \mathbf{1}$ and taking transpose of both sides we have $\mathbf{1}^\intercal H = \mathbf{1}^\intercal$ because $H$ is symmetric.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ There indeed is a column of ones. How exactly does it mean that JH = J? What I got so far: Column space means set of all possible linear combinations of X. It is a set of vectors. How can J be there if it is a matrix? en.wikipedia.org/wiki/Column_space#Definition $\endgroup$ – BCLC May 14 '15 at 15:28
  • $\begingroup$ I hope my edit answers your question. :) $\endgroup$ – bdeonovic May 14 '15 at 16:46
  • $\begingroup$ bdeonovic, why is it that 1TH = 1T? $\endgroup$ – BCLC May 14 '15 at 16:47
  • $\begingroup$ I had the property on projection matrices a bit wrong, its been corrected and added another edit to answer your second comment. $\endgroup$ – bdeonovic May 14 '15 at 16:58
  • $\begingroup$ Ugh, how do you know H1 = 1? $\endgroup$ – BCLC May 14 '15 at 17:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.