Consider a variant of the classical linear model:

$y_i=a + b\left(x_i-\bar{x} \right)+e_i $

Because $e_i \sim N \left (0, \sigma^2 \right)$, $y_i \sim N \left(a+ b \left(x_i -\bar{x} \right), \sigma^2 \right)$.

The OLS estimates of the slope and the intercept coefficient are:

$\hat{b}=\frac{\sum \left(x_i - \bar{x} \right) \left(y_i-\bar{y} \right)}{\sum \left(x_i - \bar{x} \right)^2} $ (as usual), but $\hat{a}=\bar{y}$

The two estimates, being a linear combination of normal variables are themselves normal with $\hat{b}\sim N \left( b ,\frac{ \sigma^2}{\sum \left(x_i - \bar{x} \right)^2}\right)$ and $\hat{a} \sim N \left( a, \frac{\sigma^2}{n}\right)$

I then need to show that the covariance between $\hat{a}$ and $\hat{b}$ is $0$

but I am stuck evaluating $$E \left[ \hat{a} \hat{b} \right]=\frac{1}{\sum \left( x_i-\bar{x} \right)^2} E \left[\bar{y} \sum \left(x_i -\bar{x} \right) y_i \right] $$

Could you please help me here?



1 Answer 1


For compactness, denote $X^*\equiv X - \bar X$. Denote the $n\times 2$ regressor matrix by $\mathbf Z$ (that includes a series of ones and the series of $X^*$). Then in matrix notation, the variance-covariance matrix of the $2 \times 1$ OLS estimator $\hat \beta = (\hat a, \hat b)'$ is

$$\operatorname{Var}(\hat \beta) = \sigma^2_e\left(\mathbf Z'\mathbf Z\right)^{-1}$$

Write the matrix $\mathbf Z$ in block form and you get a $1 \times 2$ block matrix. This makes easy to write out $\mathbf Z'\mathbf Z$ which will give you your answer. Signal that you're done so that I can complete this answer.

The OP found his way so:
In block form, the regressors matrix is written as a $1\times 2$ matrix

$$\left[\begin{matrix} \mathbf 1 & \mathbf x^* \end{matrix}\right]$$

where $\mathbf 1$ is a $1\times n$ column vector of ones and $\mathbf x^*$ is the $1\times n$ column containing the data on $X^*$ the centered version of $X$. Then

$$\mathbf Z'\mathbf Z =\left[\begin{matrix} \mathbf 1' \\ \mathbf x^{*'}\\ \end{matrix}\right]\left[\begin{matrix} \mathbf 1 & \mathbf x^* \end{matrix}\right] = \left[\begin{matrix} \mathbf 1'\mathbf 1& \mathbf 1'\mathbf x^{*}\\ \mathbf x^{*'}\mathbf 1 & \mathbf x^{*'}\mathbf x^{*}\\ \end{matrix}\right]$$

But $$\mathbf 1'\mathbf x^{*} = \mathbf x^{*'}\mathbf 1 = \sum_{i=1}^nx_i^* =\sum_{i=1}^n(x_i-\bar x) = 0$$

So the off-diagonal elements of $\mathbf Z'\mathbf Z$ are zero and so will be the off-diagonal elements of $\left(\mathbf Z'\mathbf Z\right)^{-1}$, i.e. of the variance-covariance matrix of the OLS estimator. But these off-diagonal elements represent the covariance between the elements of the OLS estimator. QED.

  • $\begingroup$ Indeed the off- diagonal elements vanish! Thank you! $\endgroup$
    – JohnK
    Apr 27, 2014 at 19:22
  • $\begingroup$ Hey Alecos, in view of the lack of covariance between the estimators of the slope and intercept, would you recommend this model instead of the standard OLS one or do you think it makes no difference? $\endgroup$
    – JohnK
    May 2, 2014 at 12:43
  • $\begingroup$ Let's see... Unbiasedness, consistency and asympotic normality are not affected... what about efficiency? I would suggest that you calculate analytically the variances of the estimators in the two situations and compare them (the off-diagonal elements may affect the variance through the determinant of the matrix, which is used in order to invert it). $\endgroup$ May 2, 2014 at 16:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.