Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Consider a variant of the classical linear model:

$y_i=a + b\left(x_i-\bar{x} \right)+e_i $

Because $e_i \sim N \left (0, \sigma^2 \right)$, $y_i \sim N \left(a+ b \left(x_i -\bar{x} \right), \sigma^2 \right)$.

The OLS estimates of the slope and the intercept coefficient are:

$\hat{b}=\frac{\sum \left(x_i - \bar{x} \right) \left(y_i-\bar{y} \right)}{\sum \left(x_i - \bar{x} \right)^2} $ (as usual), but $\hat{a}=\bar{y}$

The two estimates, being a linear combination of normal variables are themselves normal with $\hat{b}\sim N \left( b ,\frac{ \sigma^2}{\sum \left(x_i - \bar{x} \right)^2}\right)$ and $\hat{a} \sim N \left( a, \frac{\sigma^2}{n}\right)$

I then need to show that the covariance between $\hat{a}$ and $\hat{b}$ is $0$

but I am stuck evaluating $$E \left[ \hat{a} \hat{b} \right]=\frac{1}{\sum \left( x_i-\bar{x} \right)^2} E \left[\bar{y} \sum \left(x_i -\bar{x} \right) y_i \right] $$

Could you please help me here?


share|improve this question
up vote 4 down vote accepted

For compactness, denote $X^*\equiv X - \bar X$. Denote the $n\times 2$ regressor matrix by $\mathbf Z$ (that includes a series of ones and the series of $X^*$). Then in matrix notation, the variance-covariance matrix of the $2 \times 1$ OLS estimator $\hat \beta = (\hat a, \hat b)'$ is

$$\operatorname{Var}(\hat \beta) = \sigma^2_e\left(\mathbf Z'\mathbf Z\right)^{-1}$$

Write the matrix $\mathbf Z$ in block form and you get a $1 \times 2$ block matrix. This makes easy to write out $\mathbf Z'\mathbf Z$ which will give you your answer. Signal that you're done so that I can complete this answer.

The OP found his way so:
In block form, the regressors matrix is written as a $1\times 2$ matrix

$$\left[\begin{matrix} \mathbf 1 & \mathbf x^* \end{matrix}\right]$$

where $\mathbf 1$ is a $1\times n$ column vector of ones and $\mathbf x^*$ is the $1\times n$ column containing the data on $X^*$ the centered version of $X$. Then

$$\mathbf Z'\mathbf Z =\left[\begin{matrix} \mathbf 1' \\ \mathbf x^{*'}\\ \end{matrix}\right]\left[\begin{matrix} \mathbf 1 & \mathbf x^* \end{matrix}\right] = \left[\begin{matrix} \mathbf 1'\mathbf 1& \mathbf 1'\mathbf x^{*}\\ \mathbf x^{*'}\mathbf 1 & \mathbf x^{*'}\mathbf x^{*}\\ \end{matrix}\right]$$

But $$\mathbf 1'\mathbf x^{*} = \mathbf x^{*'}\mathbf 1 = \sum_{i=1}^nx_i^* =\sum_{i=1}^n(x_i-\bar x) = 0$$

So the off-diagonal elements of $\mathbf Z'\mathbf Z$ are zero and so will be the off-diagonal elements of $\left(\mathbf Z'\mathbf Z\right)^{-1}$, i.e. of the variance-covariance matrix of the OLS estimator. But these off-diagonal elements represent the covariance between the elements of the OLS estimator. QED.

share|improve this answer
Indeed the off- diagonal elements vanish! Thank you! – JohnK Apr 27 '14 at 19:22
Hey Alecos, in view of the lack of covariance between the estimators of the slope and intercept, would you recommend this model instead of the standard OLS one or do you think it makes no difference? – JohnK May 2 '14 at 12:43
Let's see... Unbiasedness, consistency and asympotic normality are not affected... what about efficiency? I would suggest that you calculate analytically the variances of the estimators in the two situations and compare them (the off-diagonal elements may affect the variance through the determinant of the matrix, which is used in order to invert it). – Alecos Papadopoulos May 2 '14 at 16:33

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.