0
$\begingroup$

After fitting the regression model, $$y=\beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \epsilon$$ on 15 cases, it is found that the mean square errors $$s^2 = 3$$ and

$$(X^T X)^{-1}= \begin{bmatrix} 0.5 & 0.3 & 0.2 & 0.6 \\ 0.3 & 6.0 & 0.5 & 0.4 \\ 0.2 & 0.5 & 0.2 & 0.7 \\ 0.6 & 0.4 & 0.7 & 3.0 \\ \end{bmatrix}$$

This is a practice problem for my midterm... I cannot figure out how I would figure out the estimated correlation between $$\hat\beta_1 \mbox{ and } \hat\beta_3.$$

$\endgroup$
1

1 Answer 1

2
$\begingroup$

Write $\hat{\mathbf\beta} = (\hat\beta_0,\hat\beta_1,\hat\beta_2,\hat\beta_3,)'$. You have

$$ \hat{\mathbf\beta} = (X'X)^{-1}X'y $$

You're interested in the variance-covariance matrix of $\hat{\mathbf\beta}$, which I will denote by $\text{cov}(\hat{\mathbf\beta} )$:

$$ \text{cov}(\hat{\mathbf\beta} ) = (X'X)^{-1}X'\text{var(y)}X(X'X)^{-1} = \text{var(y)}(X'X)^{-1}X'X(X'X)^{-1} = \text{var(y)}(X'X)^{-1} $$

You'll get an estimate for $\text{var(y)}$ by using $s^2$ appropriately, and you'll get the covariance between $\hat\beta_1$ and $\hat\beta_3$ by looking at the element $(2,4)$ of the covariance matrix $\text{cov}(\hat{\mathbf\beta} )$ (this is not element $(1,3)$ because the index $i$ in $\hat\beta_i$ starts at 0 for the intercept...).

From there, you can compute the correlation between $\hat\beta_1$ and $\hat\beta_3$ by standardizing appropriately, using the variances of $\hat\beta_1$ and $\hat\beta_3$, which you get from the elements $(2,2)$ and $(4,4)$ of $\text{cov}(\hat{\mathbf\beta} )$.

$\endgroup$
3
  • $\begingroup$ +1 Note, though, that the references to $s^2$ are only distracting from the ideas because the scale factor $s$ does not affect any correlations and can be ignored. $\endgroup$
    – whuber
    Commented Nov 21, 2021 at 18:02
  • $\begingroup$ Can you explain how you know which index in the matrix to look at? $\endgroup$ Commented Nov 21, 2021 at 23:29
  • $\begingroup$ Look at the dimension of the matrix $X$: it is $4\times 4$. This is because $X$ is a $15\times 4$ matrix. The confusion comes from the fact that while you have only 3 variables $x_1, x_2, x_3$, you also have a constant in the first column, associated with the coefficient $\beta_0$. So all the indices are shifted by 1.... for instance the variance of $\beta_1$ appears in $\text{cov}(\hat\beta)_{2,2}$. $\endgroup$
    – wiwh
    Commented Nov 22, 2021 at 8:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.