11
$\begingroup$

Is it true that an estimator will always asymptotically be consistent if it is biased in finite samples? I feel like it is true but not sure exactly how to prove that...

$\endgroup$
8
  • 13
    $\begingroup$ Suppose this were so. Then take any such estimator and add some arbitrary constant to it so that it remains biased in finite samples: by construction it is not asymptotically consistent. $\endgroup$
    – whuber
    Commented Dec 10, 2020 at 14:44
  • $\begingroup$ @whuber, this should be an answer, and it would arguably be the best among the ones posted. $\endgroup$ Commented Dec 11, 2020 at 8:48
  • 1
    $\begingroup$ @RichardHardy Not the whole story. Asymptotic consistency is useless without evidence of sufficient convergence. $\endgroup$
    – Carl
    Commented Dec 11, 2020 at 9:45
  • 2
    $\begingroup$ @Carl, this is beyond the OP's question. What do you mean by sufficient convergence? $\endgroup$ Commented Dec 11, 2020 at 9:48
  • $\begingroup$ @RichardHardy Sufficient for confidence in the results, as in confidence intervals, without which one is guessing as to validity. I took it upon myself to answer in the useful context, and not just lay down alms on the altar of asymptotic consistency. So, shoot me. $\endgroup$
    – Carl
    Commented Dec 11, 2020 at 9:55

4 Answers 4

26
$\begingroup$

Consider the estimator $\hat{\theta} = 3$. If this estimator is estimating a parameter that is not equal to three then it is biased in all finite samples. Is this estimator asymptotically consistent?

$\endgroup$
1
  • 1
    $\begingroup$ +1 Sometimes we forget that such simple possibilities exist. It is good to be reminded of that. $\endgroup$
    – Galen
    Commented May 7, 2022 at 17:15
6
$\begingroup$

Next to the simple and effective example of Ben, here is a more applied and specific one: Imagine you try to estimate a causal effect, but your regression model to estimate this causal parameter is misspecified (often called "omitted variable bias").

A well-known example deals with returns to schooling, i.e., how much more you earn due to (i.e., in the sense of a causal effect of sitting in class, taking courses) additional schooling. If you simply regress earnings on schooling the regression is likely misspecified, because people (after compulsory schooling ends) choose how much schooling they want. Now, more motivated and able students will, as a tendency, find the idea of going to school for longer less daunting than other students. Now, such able and motivated persons will however also be likely to be good at the workplace due to these characteristics, irrespective of how much schooling they have. Hence, they will likely earn more.

Hence, you would need to control for things like ability/motivation - which may not be easy in practice - in your regression (and likely other things, too).

Just collecting more data on your simple regression of earnings on schooling will, in turn, not save you from this problem, so both biased and inconsistent estimation. For both small and large datasets, the simple regression, as a tendency, compares earnings of students who are both able and have higher schooling to earnings of students who are less able and have less schooling. Assigning the entire difference in earnings to schooling hence will overstate the causal effect of schooling.

$\endgroup$
2
$\begingroup$

Yes, in some circumstances, no in others. For example, if a bias results from a self-inconsistent assumption, then no. Examples of this latter include omitted variable bias and AIC in the case of censored data, which violates the maximum likelihood assumption. Examples of when it pertains would be AIC in the case of complete support (i.e., without censoring such that the maximum likelihood assumption pertains), and ordinary least squares for equidistant independent axis data. In still others, for example, variance is generally unbiased, but standard deviation is not, see this. Standard deviation would still be asymptotically correct because the small number bias would reduce to zero for $n\to\infty$. Nevertheless, one should not rely on just any asymptotic convergence, if a rather better estimator is available, and see how this was done in this example. Briefly, if you small number correct standard deviations from a large number of 2 sample SD's and then average them, you will obtain a more variable estimate than if you root mean square combine all the variances and then use a much lesser small number correction for the total number of trials. Some people are surprised at how ineffectual AIC can be for small samples. Thus, how fast asymptotic convergence occurs can be critical to interpretation of statistical results, and sometimes, for example for AIC, when we do not have measures that inform us of how precise or accurate statistical results are, it can be problematic.

Thus, the question of whether or not a procedure is asymptotically convergent is not by itself a sufficient criterion of validity of statistical results. We also need confidence intervals for those results.

$\endgroup$
6
  • 1
    $\begingroup$ Did you mean "censored" rather than "censured"? $\endgroup$
    – dimitriy
    Commented Dec 10, 2020 at 17:05
  • $\begingroup$ @DimitriyV.Masterov Indeed. Nice pickup. I changed the spelling. $\endgroup$
    – Carl
    Commented Dec 11, 2020 at 0:31
  • 1
    $\begingroup$ @Ben Ha, ha. Glad I could tickle your funny-bone. At least someone, you, or should I call you "them" has a sense of humor on this depressing site. $\endgroup$
    – Carl
    Commented Dec 11, 2020 at 9:39
  • 1
    $\begingroup$ Why "them". Are you making fun of my multiple personality disorder? ; ) $\endgroup$
    – Ben
    Commented Dec 11, 2020 at 9:55
  • 1
    $\begingroup$ @Ben You are a joy, and put lightness in my heart. No, I was making fun of the web site, and "their" misplaced sense of propriety. $\endgroup$
    – Carl
    Commented Dec 11, 2020 at 10:01
0
$\begingroup$

Is it true that an estimator will always asymptotically be consistent if it is biased in finite samples?

The correct reply is trivial and it is NO, as pointed out above.

However your question immediately suggest a more interesting one:

Is it true that an estimator will always asymptotically be consistent if it is unbiased in finite samples?

The reply is: yes, if its variance going to zero when sample size diverge.

I add this part here because I suppose can be interesting for some readers.

$\endgroup$
7
  • 4
    $\begingroup$ What if the variance is decreasing but converges to a positive number rather than zero? $\endgroup$ Commented Dec 11, 2020 at 10:18
  • $\begingroup$ I Richard! In my experience I never encounter situation like this. However I suppose that no kind of consistency is achieved if the estimator variance have a lower bound greater that zero. $\endgroup$
    – markowitz
    Commented Dec 11, 2020 at 13:23
  • $\begingroup$ I think so too. Consider editing this into your answer. $\endgroup$ Commented Dec 11, 2020 at 13:30
  • $\begingroup$ ok, I made the answer it more precise. $\endgroup$
    – markowitz
    Commented Dec 11, 2020 at 13:35
  • $\begingroup$ Though what if the variance is infite but, say, the mean absolute deviation goes to zero? $\endgroup$ Commented Dec 11, 2020 at 15:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.