The OLS estimator $\hat{\beta}$ of $\beta$ is unbiased. Now, $\hat{\beta}$ is a function of the random sample (y,$x_1$,...$x_j$) and is therefore a random variable itself. When we compute $\mathbb{E}(\hat{\beta}|X)$ we are basically fixing $X$ to be the data we have and we take an expectation over the $y$ variable. Unbiasedness tells us that if we compute all possible values of $\hat{\beta}$ by runnning and OLS regression for every possible $y$, then we would find that $\mathbb{E}(\hat{\beta}-\beta|X)=0$ by averaging over the $\hat{\beta}$'s, that is, the average over all those $\hat{\beta}$'s is the true population parameter (conditional on $X$). However, in practice we have only one $y$ which will allow to compute an OLS estimate $\hat{\beta}$ which differs almost surely from $\beta$ - since $\hat{\beta}$ is random, equality would be an event of pure luck. But that being the case, why do we care so much that the coefficient is unbiased? Is it because it allows for an interpretation of the coefficients as the marginal effect on the mean of the sampled random dependent variable (or the marginal effect of $X$ on $y$ when $y$ is drawn from a typical random sample in the sense that the sample mean equals the population mean) ? I have also read coefficients being interpreted as the average marginal effect of a variable on the dependent variable. However, I would read it not as the average/mean marginal effect on the dep. variable but rather as the marginal effect on the mean of the dep.variable conditional on $X$. Which interpretation is correct?

  • 1
    $\begingroup$ Well, isn't it better (other things equal) to be right on average than wrong on average? It isn't that we care "so much" about it, it's just a nice property to have in general. That isn't to say we should never use biased estimators, oftentimes the "best" estimators are biased. $\endgroup$
    – dsaxton
    Commented Oct 10, 2016 at 4:09
  • 3
    $\begingroup$ People pay lip service to unbiasedness, but (speaking more generally than your regression example), they demonstrably don't care about it all that much. Who uses an unbiased estimator of standard deviation for example? How many people unbias their parameter estimates when fitting lognormal distributions? etc etc. I'm a bit more explicit in my opinion that many, since I don't pay much lip service to unbiasedness at all -- I really don't care much at all about unbiasedness -- it's rarely of practical relevance in things I deal with; I'll often take a biased estimate if it will get me close...ctd $\endgroup$
    – Glen_b
    Commented Oct 10, 2016 at 5:43
  • $\begingroup$ ctd... to what I want. Other things being equal, of course no bias is better than bias, but so often other things are far from equal. $\endgroup$
    – Glen_b
    Commented Oct 10, 2016 at 5:45
  • $\begingroup$ @Glen_b: but in the context of regression coefficients it may be of importance, e.g. if you perform hypothesis tests on the coefficient or if you compute a confidence interval for the coefficent ? For hypothesis tests you test whether the ''true'' value is e.g. different from zero (I think it is good to work with an unbiased estimate in that case), similar for a confidence interval, you find a (random) interval that has a certain probability to contain the ''true'' value ? With biased estimates you have tests/intervals for the ''true'' value plus ''something'' ? $\endgroup$
    – user83346
    Commented Oct 10, 2016 at 6:44
  • $\begingroup$ @fcop Hypothesis tests are case in point -- many people use likelihood ratio tests but most MLEs are biased. Similarly many confidence intervals are not centered on the parameter. Consider confidence intervals for the standard deviation of the error term in regression for example. Or if you really want to focus on regression coefficients, consider the case where you have a contaminating process that leads to occasional wild values that are not centered at the underlying uncontaminated population line we're interested in. A good estimator will clearly be biased for the ...ctd $\endgroup$
    – Glen_b
    Commented Oct 10, 2016 at 7:14


Browse other questions tagged or ask your own question.