1
$\begingroup$

I'm having some trouble understanding the following expression:

$E(\bar{u}|x_i)=0$

$u$ is the error term and $\bar{u}=n^{-1}\sum u$

It's used for proving unbiasedness of OLS.

I've got two questions:

Does $\bar{u}$ equal zero like for the residuals? I know that the expected value of u equals zero.

If yes, then taking expectation of $\bar{u}$ is like taking expectation of a constant and therefore the expected value is redundant?

$\endgroup$
8
  • 1
    $\begingroup$ Shall we presume that "error term" means in the model and is not the residual with respect to the OLS fit? (This is the conventional meaning, but many people are confused by the difference.) If so, shall we also presume that you consider your data to be a sample where the $u_i$ are independent? If these assumptions hold, then could you explain why you think the $u_i$ must sum to zero? $\endgroup$
    – whuber
    Sep 23, 2016 at 17:37
  • 1
    $\begingroup$ Yes, I'm talking about the error term of the population model, not the residual $\hat{u}$. For proving unbiasedness of $\hat{\beta}_0$, I had to take the average of the population regression model written in terms of a random sample. What's important to point out is, that $E(\bar{u})$ is conditional on the $x_i$ values. Therefore the $u_i$ are not related to $x_i$ to prove unbiasedness. I think the $u_i$ must sum to zero because the residuals sum to zero. I'm a bit confused because the expected value of $u$ equals zero, but I believe the average isn't quite the same. $\endgroup$
    – S. Ming
    Sep 23, 2016 at 17:51
  • 1
    $\begingroup$ Let's look at the situation as simply as possible, abstracting away the inessential features. You have a collection of random variables $u_i$ and you assume they are independent. If they had to sum to zero, then the last one would be mathematically determined by the first $n-1$ of them. They could hardly be independent then, could they? At any rate, if you could edit your post to incorporate the additional information in your comments, then it ought to be clear enough to be answerable. $\endgroup$
    – whuber
    Sep 23, 2016 at 18:05
  • $\begingroup$ I'm a beginner at stats and I'm having a hard time to grasp certain concepts;) I understand that the $u_i$ can't be independent when they have to sum to zero. But if we would know the whole population (hypothetically) and we would run a regression, then the $u_i$ would add up to zero. Therefore I think that the average of $u_i$ is zero. Correct? $\endgroup$
    – S. Ming
    Sep 23, 2016 at 18:20
  • $\begingroup$ Let's consider a very simple example. Suppose $n=2$ and each $u_i$ is either $1$ or $-1$ with equal probability, so that the expected value of each one of them is zero. That is, you have a coin with "+1" written on one side and "-1" on the other and you flip this coin twice. Is it guaranteed that the sum of the coins, $u_1+u_2$, is zero? Noting that $\bar u = (u_1+u_2)/2$ is proportional to the sum, what does that tell you about their mean $\bar u$? $\endgroup$
    – whuber
    Sep 23, 2016 at 19:15

1 Answer 1

1
$\begingroup$

The main issue here is that you are confused. Let us look at this in the setting of simple linear regression, the model is $$ Y_i=\beta_0 + \beta_1 x_i+u_i, \quad i=1,\dotsc n $$ (You didn't write a model equation in the question, doing that is often a first step in clearing things up.) For instance, you write only $u$ and not the more correct and informative $u_i$. Under the usual OLS assumptions we have $u_i$ are iid (independent identically distributed) with mean 0 and variance $\sigma^2$. Then we have $$ \DeclareMathOperator{\E}{\mathbb{E}} \E \bar{u}=\E \frac1{n}\sum_1^n u_i =\frac1{n} \sum_1^n \E u_i =0 $$ But then in a comment you say:

If we'd know the true regression line, wouldn't the $∑u_i$ equal to zero?

But, even if you know the true values of $\beta_0, \beta_1$, the error terms $u_i$ are still independent random variables as above, and there is no reason their sum is identically zero. That is not consistent with independence (and positive variance.)

It is not clear what you mean by data of the whole population so I will not comment on that part.

$\endgroup$
5
  • $\begingroup$ What’s not making sense to me is how the mean can be zero without the sum being zero. I know it has to do with the expectation, but maybe you could expand on this. $\endgroup$
    – Dave
    May 5, 2020 at 4:20
  • 2
    $\begingroup$ The $u_i$, the errors, are theoretical terms. They are unobserved random variables, so is their sum. In one realization, the sum could be positive, in another one it could be negative. You would not know. But in the long term, over many realizations, it would even out. $\endgroup$ May 5, 2020 at 4:35
  • $\begingroup$ @Dave The mean of a fair coin that has the values $\pm 1$ on its faces is zero. Flip the coin three times: I guarantee the mean of those three values will not be zero! $\endgroup$
    – whuber
    May 22 at 12:40
  • $\begingroup$ @whuber But then neither is the sum. $\endgroup$
    – Dave
    May 22 at 12:56
  • $\begingroup$ @Dave Exactly: and therein lies the distinction between the expectation and the actual value of a random variable, which I suspect might be one of the key issues here. $\endgroup$
    – whuber
    May 22 at 13:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.