I was wondering if it would be possible to get the correlation between $\hat\beta_0$ and $\hat\beta_1$ in the linear regression model $y_i=\beta_0+\beta_1x_i+\varepsilon_i$. If so, how should I derive such an expression? (Let's assume $\varepsilon_i$ ~ independent with mean 0 and finite variance > 0.) Since I am not very used to advanced statistics, a detailed explanation would be very helpful.


The estimates $\boldsymbol{\hat{\beta}}=( \hat{\beta_0}, \hat{\beta_1}$) are given by formula $\boldsymbol{\hat{\beta}}=(X'X)^{-1}X'Y$. Variance of this vector of estimates is $\boldsymbol{\hat{\beta}}=\sigma^2(X'X)^{-1}$, where $X$ is a matrix with two columns - first of ones, the second of $x_i$, and $\sigma^2$ is the variance of errors. The $(X'X)^{-1}$ matrix is given by:

$(X'X)^{-1} = \left( \begin{array}{ccc} n & \sum x_i \\ \sum x_i & \sum x_i^2 \end{array} \right)^{-1}= \frac{1}{det (X'X)} \left( \begin{array}{ccc} \sum x_i^2 & -\sum x_i \\ -\sum x_i & n \end{array} \right)=\frac{1}{n\sum x_i^2 - (\sum x_i)^2 }\left( \begin{array}{ccc} \sum x_i^2 & -\sum x_i \\ -\sum x_i & n \end{array} \right)$

Correlarion between the two estimates should then be $\frac{-\sum x_i}{\sqrt{n\sum x_i^2}}$.

  • $\begingroup$ You mean that the variance of the vector β hat is (x'x)^-1, right? And how did you derive the (X′X)^−1 matrix? $\endgroup$
    – Betty
    Nov 2 '14 at 2:52
  • $\begingroup$ The variance of the vector is the matrix you wrote times the variance of errors. I will edit the answer, so that it shows in LaTeX. $\endgroup$
    – DatamineR
    Nov 2 '14 at 3:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.