0
$\begingroup$

I would like to understand the interpretation of $det(X'X)$ in case of multiple regressors.

$Var(x) = \sum_i^n(x_i-\bar{x})^2 = \frac{1}{n}\sum_i^nx_i^2 - \bar{x}^2 = \frac{1}{n}\sum_i^nx_i^2 - \frac{(\sum_i^n x_i)^2}{n^2}$

SLR case: $\hat{y_i} = \hat{\beta_0} + \hat{\beta_1}x_i$

$\beta = (X'X)^{-1}X'y$

$X'X =\begin{bmatrix} n & \sum_i^nx_i\\ \sum_i^nx_i & \sum_i^nx_i^2 \end{bmatrix}$

$(X'X)^{-1} =\frac{1}{n\sum_i^nx_i^2 - (\sum_i^nx_i)^2}\begin{bmatrix} \sum_i^nx_i^2 & -\sum_i^nx_i\\ -\sum_i^nx_i & n \end{bmatrix} = \frac{n}{n^2Var(x)}\begin{bmatrix} \frac{\sum_i^nx_i^2}{n} & -\bar{x}\\ -\bar{x} & 1 \end{bmatrix}$

Thus, if I get it right, in SLR case $det(X'X) = n^2Var(x)$

Now, let's consider case with two regressors, but without intercept, then $(X'X)$ is still 2x2.

$\tilde{y_i} = \tilde{\beta_1}x_i + \tilde{\beta_2}x_i$

$X'X =\begin{bmatrix} \sum_i^nx_{1i}^2 & \sum_i^nx_{1i}x_{2i}\\ \sum_i^nx_{1i}x_{2i}& \sum_i^nx_{2i}^2 \end{bmatrix}$

$(X'X)^{-1} =\frac{1}{\sum_i^nx_{1i}^2\sum_i^nx_{2i}^2 - \sum_i^nx_{1i}\sum_i^nx_{2i}}\begin{bmatrix} \sum_i^nx_{2i}^2 & -\sum_i^nx_{1i}x_{2i}\\ -\sum_i^nx_{1i}x_{2i} & \sum_i^nx_{1i}^2 \end{bmatrix}$

Can someone please provide the interpretation of $det(X'X)$ in the case of two regressors? I think that it should be linked with variances of $x_{1i}, x_{2i}$ and their covariance, but I don't have a solid understaning. Also, it will be great to get insights about more general case of several regressors with intercept.

$\endgroup$
  • 1
    $\begingroup$ There are various interpretations possible. The one you seem to be looking for might be closely related to the account I gave of the connection between the covariance matrix and the Normal equations of multiple regression at stats.stackexchange.com/a/108862/919. $\endgroup$ – whuber May 8 at 17:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.