3
$\begingroup$

This is an excerpt from the second edition of An Introduction to Generalized Linear Models by Annette J. Dobson:

enter image description here enter image description here

The author seems to assume that $(I - H)/\sigma^2 = V^-$, where $V$ is the variance-covariance matrix for $y$, and $V^-$ the generalized inverse, it's not clear how he arrived there.

The author referred to section 1.4.2 part 8, it's also listed here:

enter image description here

$\endgroup$
  • $\begingroup$ (i) which book? (ii) I don't see the author assuming that there. Presumably you have reasoned your way to saying that. Can you fill in the details of how you arrived at that conclusion about what the author assumes? $\endgroup$ – Glen_b -Reinstate Monica Jan 4 '14 at 22:18
  • 1
    $\begingroup$ I linked to the relevant page of the book, for it is certainly useful to know the context and notation used, e.g. that D is the deviance. $\endgroup$ – Hibernating Jan 5 '14 at 4:13
  • 1
    $\begingroup$ @Glen_b I have edited the question. $\endgroup$ – qed Jan 5 '14 at 10:08
3
$\begingroup$

The OP is right in concluding that the author, for internal consistency, has to treat $\frac 1{\sigma^2}\mathbf M=\frac 1{\sigma^2}(\mathbf I - \mathbf H)$ as a generalized inverse of the variance covariance matrix of $\mathbf y$, to obtain the result the author arrives at (of zero non-centrality parameter).

One page previously in the book (we are at section 5.6.2), the distribution of $\mathbf y$ is assumed as

$$\mathbf y \sim N(\mathbf X\beta, \sigma^2\mathbf I)$$ so the question is: does $\frac 1{\sigma^2}\mathbf M$ satisfy the required properties so that it is a generalized inverse of $\sigma^2\mathbf I$?

These properties are four (see for example here), and it is easily seen that $\frac 1{\sigma^2}\mathbf M$, due to it being symmetric and idempotent, satisfies the 2nd, 3d and 4th -but it does not satisfy the 1st. We have:

1st : $\left(\sigma^2\mathbf I\right)\left(\frac 1{\sigma^2}\mathbf M\right)\left(\sigma^2\mathbf I\right) = \sigma^2 \mathbf M \neq \sigma^2 \mathbf I \qquad \text{not satisfied}$

2nd : $\left(\frac 1{\sigma^2}\mathbf M\right)\left(\sigma^2\mathbf I\right)\left(\frac 1{\sigma^2}\mathbf M\right) = \frac 1{\sigma^2} \mathbf M \mathbf M = \frac 1{\sigma^2} \mathbf M \qquad \text{satisfied}$

3d : $\left(\sigma^2\mathbf I\frac 1{\sigma^2}\mathbf M\right)^T = \left(\frac 1{\sigma^2}\mathbf M^T\right)\left(\sigma^2\mathbf I^T\right) = \left(\sigma^2\mathbf I\right)\left(\frac 1{\sigma^2}\mathbf M\right) \qquad \text{satisfied}$

4th : $\left(\frac 1{\sigma^2}\mathbf M\sigma^2\mathbf I\right)^T = \left(\sigma^2\mathbf I^T\right)\left(\frac 1{\sigma^2}\mathbf M^T\right) = \left(\frac 1{\sigma^2}\mathbf M\right)\left(\sigma^2\mathbf I\right) \qquad \text{satisfied}$

The problem is that the 1st condition is the fundamental property in order for a matrix to be a generalized inverse (in fact, if just the 1st is satisfied we have a generalized inverse -the rest of the conditions are needed to successively obtain generalized inverses endowed with more properties, leading to the Moore-Penrose unique pseudo-inverse that satisfies all four conditions).

So it appears that $\frac 1{\sigma^2}\mathbf M$ cannot be treated as a generalized inverse of $\frac 1{\sigma^2}\mathbf I$, and so the "zero non-centrality parameter" result cannot follow from section 1.4.2 part 8 of the book as the author claims. Now, A. Dobson is not usually wrong, so maybe I am missing something, but I am posting this answer so that perhaps somebody else can settle this better...

(ADDENDUM)
...and @whuber just did! We have (regressors are deterministic here)

$$\mathbf y \sim N(\mathbf X\beta, \sigma^2\mathbf I) \Rightarrow \mathbf z=\mathbf M\mathbf y\sim N(\mathbf 0, \sigma^2\mathbf M)$$

Note the variance-covariance matrix of $\mathbf z$ is $V(\mathbf z) = \mathbf MV(\mathbf y)\mathbf M^{T} = \sigma^2 \mathbf M $

where we have used the properties of $\mathbf M$, $\mathbf M\mathbf X=\mathbf 0$ and $\mathbf M\mathbf M^T = \mathbf M\mathbf M=\mathbf M$. Then, we have that $\mathbf z^T(\sigma^2\mathbf M)^-\mathbf z$ follows a central chi-square. But $$\mathbf z^T(\sigma^2\mathbf M)^-\mathbf z = \mathbf y^T\mathbf M^T(\sigma^2\mathbf M)^-\mathbf M\mathbf y$$

and $\mathbf M$ can be used as a generalized inverse of its own self because $\mathbf M\mathbf M \mathbf M =\mathbf M$. So

$$\mathbf z^T(\sigma^2\mathbf M)^-\mathbf z = \mathbf y^T\mathbf M^T\frac 1 {\sigma^2} \mathbf M\mathbf M\mathbf y = \mathbf y^T\left(\frac 1 {\sigma^2}\mathbf M\right)\mathbf y $$

Therefore $D =\mathbf y^T\left(\frac 1 {\sigma^2}\mathbf M\right)\mathbf y$ follows a central chi-squared.

$\endgroup$
  • $\begingroup$ I'm missing something: you appear to be checking whether $M$ is a multiple of a generalized inverse of $I$, the identity matrix. Of course $I$ has a unique generalized inverse, and it certainly isn't equal to $I-H$! I don't see how this check is relevant to anything in the question. $\endgroup$ – whuber Jan 6 '14 at 17:19
  • $\begingroup$ @Whuber But this is exactly the question here : is $\mathbf M$ a generalized inverse of the variance of $\mathbf y$ (ignoring the constant), so that we can invoke section 1.4.2. part 8 of the book? This is the reasoning used in the book excerpt: "the deviance can be written as $\mathbf y^T\mathbf M\mathbf y$, and if $\mathbf M$ is a generalized inverse of the variance of $\mathbf y$ then by section 1.4.2 part 8, we know that $\mathbf y^T\mathbf M\mathbf y$ follows a non-central chi-square" etc. My answer points that it does not appear that we can use this chain of reasoning. $\endgroup$ – Alecos Papadopoulos Jan 6 '14 at 17:27
  • 4
    $\begingroup$ I believe the book is not applying these theorems quite as directly as you are. The relevant application is to $(I-H)y$ rather than to $y$ itself. The stated results follow immediately. $\endgroup$ – whuber Jan 6 '14 at 17:29
  • $\begingroup$ You missed something in the second last line. $\endgroup$ – qed Jan 6 '14 at 20:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.