0
$\begingroup$

Consider linear regression. It is known that if $Y \sim N_n\left(X\beta, \sigma^2 I_n\right)$, where $X$ is $n \times p$ of rank $p$, then $$ \hat{\beta} \sim N_p\left(\beta, \sigma^2(X^TX)^{-1}\right) $$ (Seber, George A. F. and Lee, Alan J., "Linear Regression Analysis", 2nd Edition, 2003, John Wiley & Sons, Theorem 3.5 (i), p. 47.)

If I understand correctly, $n$ is the number of observations, and each observation consists of $p$ components. If this is the case, I would expect that the order by which the observations are listed should have no effect on $\hat{\beta}$'s distribution. And yet, if $Z$ is an $n\times p$ matrix that is obtained from $X$ by permuting $X$'s rows (i.e. by listing the observations in a different order), $Z^TZ$ is not necessarily the same as $X^TX$, and hence the two $\hat{\beta}$'s will not necessarily be equally distributed.

How can it be?

$\endgroup$
4
$\begingroup$

And yet, if $Z$ is an $n×p$ matrix that is obtained from $X$ by permuting $X$'s rows (i.e. by listing the observations in a different order), $Z^TZ$ is not necessarily the same as $X^TX$

Can you show some examples where your claim that they can differ is true?

Note that the $i,j$ element of $X^TX$ is $\sum_k x_{k,i}x_{k,j}$. This sum won't change when you alter the order of the $k$'s you're summing over.

You might try a few small examples and see what effect permutation of the rows of $X$ has.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.