2
$\begingroup$

"the numerator expresses a measure of squared distance standardized by the covariance matrix"

That is the author's interpretation on page 97 in Introduction to Linear Regression Analysis by Montgomery, Peck, and Vining.

I'm trying to understand it.

For the usual multiple linear regression model/assumptions $y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + ... \beta_k x_k +\epsilon$, or $\mathbf{y=X\beta + \epsilon}$, my text book introduces the general linear hypothesis as $H_0: \mathbf{ T \beta = c}$.

For example, if $k=4$ and $H_0: \beta_1=\beta_3,\beta_2=7$, then $\bf T=\begin{bmatrix}0&1&0&-1&0\\0&0&1&0&0\end{bmatrix}$ and $\mathbf c = \begin{bmatrix}0 & 7\end{bmatrix}^{\prime}$

For the least squares estimate $\mathbf b$ of $\beta$ and $\mathbf T$ of rank $r$, the F-statistic is given as $F_0 = \frac{(\mathbf{T b -c})^{\prime}[\mathbf{T(X^{\prime}X)^{-1}T^{\prime}}]^{-1}(\mathbf{T b -c})/r}{SS_{res}({\tiny FullModel})/(n-k-1)}$

Assuming I mathjax'd that correctly, the above is basic standard multiple linear regression.

Shortly after giving $F_0$, the author comments: "Notice that the numerator expresses a measure of squared distance between $\mathbf{T \beta}$ and $\mathbf c$ standardized by the covariance matrix of $\mathbf{T b}$".

I know $\sigma^2\mathbf{(X^{\prime}X)^{-1}}$ is the covariance matrix of $\mathbf b$. I understand what it means to 'standardize'. And I "understand" the rest.

But on my own, I would not be able to interpret $(\mathbf{T b -c})^{\prime}[\mathbf{T(X^{\prime}X)^{-1}T^{\prime}}]^{-1}(\mathbf{T b -c})/r$ as being "a measure of squared distance standardized by the covariance matrix". Nor vice-versa: if I were given the english sentence, I would not be able formulate the numerator.

Can someone break it down? and ... Is a $\sigma^2$ missing?

$\endgroup$
  • $\begingroup$ I believe $\sigma^2$ is not missing. It's invisibly present in both the numerator and the denominator, so it cancels. Compare it with equation (3.40) from which this is derived, and with Appendix C.3.4, which comes close to showing what you want. $\endgroup$ – whuber Nov 16 '15 at 0:31
  • $\begingroup$ Thanks for looking into this. I'm still clueless on the author's interpretation. Regarding eqn 3.40, the author states without derivation that $SS_H = (\mathbf{T b -c})^{\prime}[\mathbf{T(X^{\prime}X)^{-1}T^{\prime}}]^{-1}(\mathbf{T b -c})$. I'm not following you regarding Appendix C3.4 explaining the phrase "a measure of squared distance standardized by the covariance matrix". From what I've been reading, it seems the interpretation is based on quadratic forms, which I'd need to learn about from scratch, unless someone is kind enough to shortcut it here. $\endgroup$ – cwackers Nov 16 '15 at 2:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.