I was wondering if it would be possible to get the correlation between $\hat\beta_0$ and $\hat\beta_1$ in the linear regression model $y_i=\beta_0+\beta_1x_i+\varepsilon_i$. If so, how should I derive such an expression? (Let's assume $\varepsilon_i$ ~ independent with mean 0 and finite variance > 0.) Since I am not very used to advanced statistics, a detailed explanation would be very helpful.
$\begingroup$
$\endgroup$
$\begingroup$
$\endgroup$
The estimates $\boldsymbol{\hat{\beta}}=( \hat{\beta_0}, \hat{\beta_1}$) are given by formula $\boldsymbol{\hat{\beta}}=(X'X)^{-1}X'Y$. Variance of this vector of estimates is $\boldsymbol{\hat{\beta}}=\sigma^2(X'X)^{-1}$, where $X$ is a matrix with two columns - first of ones, the second of $x_i$, and $\sigma^2$ is the variance of errors. The $(X'X)^{-1}$ matrix is given by:
$(X'X)^{-1} = \left( \begin{array}{ccc} n & \sum x_i \\ \sum x_i & \sum x_i^2 \end{array} \right)^{-1}= \frac{1}{det (X'X)} \left( \begin{array}{ccc} \sum x_i^2 & -\sum x_i \\ -\sum x_i & n \end{array} \right)=\frac{1}{n\sum x_i^2 - (\sum x_i)^2 }\left( \begin{array}{ccc} \sum x_i^2 & -\sum x_i \\ -\sum x_i & n \end{array} \right)$
Correlarion between the two estimates should then be $\frac{-\sum x_i}{\sqrt{n\sum x_i^2}}$.
-
$\begingroup$ You mean that the variance of the vector β hat is (x'x)^-1, right? And how did you derive the (X′X)^−1 matrix? $\endgroup$ – Betty Nov 2 '14 at 2:52
-
$\begingroup$ The variance of the vector is the matrix you wrote times the variance of errors. I will edit the answer, so that it shows in LaTeX. $\endgroup$ – DatamineR Nov 2 '14 at 3:24
