So my professor used the following so show that using averages instead of the full data set in OLS Regression will not cause any bias in our estimator. I understand everything up to the penultimate equality, as that would imply that Cov(X_id, X_d_bar) = Var(X_bar), which is either wrong or I'm really missing something. I tried for more than an hour to understand this and I'm sure that there must be some obvious thing that im just no seeing.

enter image description here

  • $\begingroup$ Does "$\bar X_d$" mean $(1/n)\sum_{i=1}^nX_{id}$? If so plug this in to evaluate both the numerator and denominator in question. It helps to exploit the bilinearity of covariance. $\endgroup$
    – whuber
    Jan 20 at 23:24
  • $\begingroup$ Yes, I tried this many times to just substitute and rewrite it, but i always end up with Cov(X_id, X_bar_d) = Var(X_id) =!= Var(X_bar_id) $\endgroup$ Jan 20 at 23:30
  • 1
    $\begingroup$ $\text{Cov}(X_{id},\bar X_d) =\frac 1n \text{Var}( X_{id})= \text{Var}(\bar X_d)$ $\endgroup$
    – Henry
    Jan 20 at 23:41

1 Answer 1


We want to show $Cov(X_{id},\bar{X}_d) = Var(\bar{X}_d)$.

Writing things out explicitly (make sure you follow each step and understand what properties of iid data, covariances, and variances we are using in each step):

$$Cov(X_{id},\bar{X}_d) = Cov\bigg(X_{id}, \frac{1}{n}\sum_{j=1}^n X_{jd}\bigg).$$

We know that $Cov(X,aY) = aCov(X,Y)$ so

$$Cov\bigg(X_{id}, \frac{1}{n}\sum_{j=1}^n X_{jd}\bigg) = \frac{1}{n}Cov\bigg(X_{id},\sum_{j=1}^n X_{jd}\bigg).$$

We also know that $Cov(X,Y+W) = Cov(X,Y) + Cov(X,W)$ so that

$$\frac{1}{n}Cov\bigg(X_{id},\sum_{i=1}^n X_{id}\bigg) = \frac{1}{n}\sum_{j=1}^n Cov(X_{id},X_{jd}).$$

Finally, since data is i.i.d., $Cov(X_{id},X_{jd}) = 0$ for $i\neq j$, so that

$$\frac{1}{n}\sum_{j=1}^n Cov(X_{id},X_{jd}) = \frac{1}{n}Cov(X_{id},X_{id}).$$

By definition, $Cov(X,X) = Var(X)$, so we showed that $Cov\bigg(X_{id}, \frac{1}{n}\sum_{j=1}^n X_{jd}\bigg) = \frac{1}{n}Var(X_{id})$. So it remains to show that $\frac{1}{n}Var(X_{id}) = Var(\bar{X}_d)$, which follows from

$$Var(\bar{X}_d)= Var\bigg(\frac{1}{n}\sum_{i=1}^n X_{id}\bigg) = \frac{1}{n^2}Var\bigg(\sum_{i=1}^n X_{id}\bigg) = \frac{1}{n^2} \sum_{i=1}^n Var(X_{id}) = \frac{1}{n}Var(X_{id}),$$ where the third equality follows because $X_{id}$ are independent and the last equality follows because all $X_{id}$ are identically distributed so that $\sum_{i=1}^n Var(X_{id}) = n Var(X_{id})$.

  • 2
    $\begingroup$ Alright, so now i finally see what mistake I kept making. I somehow failed to consider that obviously $Cov(X_{jd}, X_{id}) = 0 \forall j \neq i$. Thanks for clearing that up for me. $\endgroup$ Jan 21 at 0:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.