0
$\begingroup$

I try to prove one result from linear regression model as follows.

Let $\beta_j$ denote the $j$th element of $\beta$ and $A_{ii}^{-1}$ the $i$th diagonal element of $A^{-1}$. Then we have $$\frac{\hat{\beta}_j-\beta_j}{\sqrt{(X^TX)^{-1}_{jj}\hat{\sigma}^2}}\sim t_{n-p} $$ where $$ \hat{{\sigma}^2}=\frac{1}{n-p}(Y-\hat{Y})^T(Y-\hat{Y})$$

Indeed, $\hat{\beta}-\beta\sim N(0, \sigma^2(X^TX)^{-1})$ and is independent of $\hat{\sigma}^2$. Then $$\frac{\hat{\beta}-\beta}{\hat{\sigma}}\sim\frac{\hat{\beta}-\beta}{\sqrt{\frac{\sigma^2\chi^2_{n-p}}{n-p}}}\sim \frac{N(0, (X^TX)^{-1})}{\sqrt{\frac{\chi^2_{n-p}}{n-p}}} $$

But I am confused that why $$ \frac{N(0, (X^TX)^{-1})}{\sqrt{(X^TX)^{-1}_{jj}}}\sim N(0,I)? $$ Isn't it divided by $\sqrt{(X^TX)^{-1}}$?

$\endgroup$
4
  • 1
    $\begingroup$ $$\frac{N(0, (X^TX)^{-1})}{\sqrt{(X^TX)^{-1}_{jj}}}\sim N(0,I)$$ This makes no sense to me. In the numerator and right hand side you have a matrix as variance. Do you refer to a multivariate normal distribution? $\endgroup$ Jan 9 at 17:10
  • 2
    $\begingroup$ You have mixed up vectors and scalars. $\endgroup$ Jan 9 at 18:49
  • $\begingroup$ @SextusEmpiricus But if we want to show that is t-distribution, we need to the numerator is standard normal. $\endgroup$
    – Hermi
    Jan 10 at 0:49
  • $\begingroup$ We do not "divide" by a matrix, we multiply by its inverse. $\endgroup$
    – usεr11852
    Jan 10 at 2:10

1 Answer 1

2
$\begingroup$

Isn't it divided by $\sqrt{(X^TX)^{-1}}$?

$\sqrt{(X^TX)^{-1}}$ is not a single number.

This is because $X$ is a $n$ by $p$ matrix and $X^T X$ will be a $p$ by $p$ matrix of which you need the $j$-th diagonal element.

But I am confused that why $$ \frac{N(0, (X^TX)^{-1})}{\sqrt{(X^TX)^{-1}_{jj}}}\sim N(0,I)?$$

This is not the case.

  • Yes, you do have that each individual $\hat\beta_j - \beta_j$ is normal distributed

  • But no, the multivariate case does not hold. The scaled $\hat\beta_j - \beta_j$ will be correlated.

In addition the expression $$\frac{N(0, (X^TX)^{-1})}{\sqrt{(X^TX)^{-1}_{jj}}}$$ does not seem quite right. The arithmetics/operation, how you divide the multivariate normal distribution, is not very clear and defined.

$\endgroup$
9
  • $\begingroup$ My question is that how to show that $\frac{N(0, (X^TX)^{-1})}{\sqrt{(X^TX)^{-1}_{jj}}}\sim N(0,I)? $ $\endgroup$
    – Hermi
    Jan 10 at 0:47
  • $\begingroup$ @quasAliki I responded to your sentence that ended in a question mark. You can not divide by the product $X^TX$ because it is a matrix. $\endgroup$ Jan 10 at 0:58
  • $\begingroup$ Regarding your question why we have $$\frac{N(0, (X^TX)^{-1})}{\sqrt{(X^TX)^{-1}_{jj}}}\sim N(0,I)?$$ The answer is that we do not have this. The expression makes no sense. Or at least for me it is unclear what you mean by the expression $N(0, (X^TX)^{-1})$ which is not a typical expression. $\endgroup$ Jan 10 at 1:00
  • $\begingroup$ If not, how to prove that $\frac{\hat{\beta}_j-\beta_j}{\sqrt{(X^TX)^{-1}_{jj}\hat{\sigma}^2}}\sim t_{n-p} $? $\endgroup$
    – Hermi
    Jan 10 at 1:07
  • $\begingroup$ @quasAliki in your question you use $$\hat{\beta}-\beta\sim N(0, \sigma^2(X^TX)^{-1})$$ that is a multivariate distribution. For the derivation/proof you can use the distribution of only the $j$-th component of that distribution $$\hat{\beta}_j-\beta_j \sim N(0, \sigma^2(X^TX)_{jj}^{-1})$$ $\endgroup$ Jan 10 at 1:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.