# Multiple Linear Regression and Correlation of two beta estimates

After fitting the regression model, $$y=\beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \epsilon$$ on 15 cases, it is found that the mean square errors $$s^2 = 3$$ and

$$(X^T X)^{-1}= \begin{bmatrix} 0.5 & 0.3 & 0.2 & 0.6 \\ 0.3 & 6.0 & 0.5 & 0.4 \\ 0.2 & 0.5 & 0.2 & 0.7 \\ 0.6 & 0.4 & 0.7 & 3.0 \\ \end{bmatrix}$$

This is a practice problem for my midterm... I cannot figure out how I would figure out the estimated correlation between $$\hat\beta_1 \mbox{ and } \hat\beta_3.$$

• Welcome to CV. If this question relates to a class exercise, please see stats.stackexchange.com/tags/self-study/info and add the tag to modify the question accordingly. Nov 21, 2021 at 17:47

Write $$\hat{\mathbf\beta} = (\hat\beta_0,\hat\beta_1,\hat\beta_2,\hat\beta_3,)'$$. You have

$$\hat{\mathbf\beta} = (X'X)^{-1}X'y$$

You're interested in the variance-covariance matrix of $$\hat{\mathbf\beta}$$, which I will denote by $$\text{cov}(\hat{\mathbf\beta} )$$:

$$\text{cov}(\hat{\mathbf\beta} ) = (X'X)^{-1}X'\text{var(y)}X(X'X)^{-1} = \text{var(y)}(X'X)^{-1}X'X(X'X)^{-1} = \text{var(y)}(X'X)^{-1}$$

You'll get an estimate for $$\text{var(y)}$$ by using $$s^2$$ appropriately, and you'll get the covariance between $$\hat\beta_1$$ and $$\hat\beta_3$$ by looking at the element $$(2,4)$$ of the covariance matrix $$\text{cov}(\hat{\mathbf\beta} )$$ (this is not element $$(1,3)$$ because the index $$i$$ in $$\hat\beta_i$$ starts at 0 for the intercept...).

From there, you can compute the correlation between $$\hat\beta_1$$ and $$\hat\beta_3$$ by standardizing appropriately, using the variances of $$\hat\beta_1$$ and $$\hat\beta_3$$, which you get from the elements $$(2,2)$$ and $$(4,4)$$ of $$\text{cov}(\hat{\mathbf\beta} )$$.

• +1 Note, though, that the references to $s^2$ are only distracting from the ideas because the scale factor $s$ does not affect any correlations and can be ignored.
– whuber
Nov 21, 2021 at 18:02
• Can you explain how you know which index in the matrix to look at? Nov 21, 2021 at 23:29
• Look at the dimension of the matrix $X$: it is $4\times 4$. This is because $X$ is a $15\times 4$ matrix. The confusion comes from the fact that while you have only 3 variables $x_1, x_2, x_3$, you also have a constant in the first column, associated with the coefficient $\beta_0$. So all the indices are shifted by 1.... for instance the variance of $\beta_1$ appears in $\text{cov}(\hat\beta)_{2,2}$.
– wiwh
Nov 22, 2021 at 8:46