1
$\begingroup$

I understand that in case of consistent estimates, larger the sample size, there's a higher probability that the estimate converges to true value of parameter. Now, using the sufficient condition of consistency, which has asymptotic unbiasedness as a condition, can I say that bias decreases as sample size increases?

OR

Since unbiasedness is a finite sample property unlike consistency, bias cannot be related to sample size?

$\endgroup$
6
  • 1
  • $\begingroup$ Okay. A biased estimate can be consistent. But as sample size increase, will the bias decrease? $\endgroup$ – Harry Feb 19 '20 at 10:20
  • $\begingroup$ Usually, the bias is a given of an estimator and not conditional on sample size. As the sample size increases, your estimate will converge to the true value if the bias is zero, it will converge to some biased value if not. The sample size affects the standard error of your empirical estimate. In most cases the inherent bias of the estimate will not change if you increase $n$, it is a given, but you have to decompose your estimator in terms of the variance and bias and evaluate whether the former is a function of $n$ see $\endgroup$ – Mark Verhagen Feb 19 '20 at 10:42
  • $\begingroup$ I'd have thought what would be more striking in those threads would be the assertion that a consistent estimator needn't be asymptotically unbiased, contrary to a premiss of your question. And the accepted answer in the first gives an example of a consistent estimator whose bias remains constant as sample size increases $\endgroup$ – Scortchi - Reinstate Monica Feb 19 '20 at 10:48
  • $\begingroup$ @MarkVerhagen: "Usually" & "in most cases" allow for some slack, but only-asymptotically-unbiased estimators are very commonly used, e.g. in the use of sample standard deviation with Bessel's correction as an estimator of population standard deviation for the Normal distribution, or in estimation of the parameters of popular generalized linear models by maximum-likelihood. $\endgroup$ – Scortchi - Reinstate Monica Feb 19 '20 at 11:14
1
$\begingroup$

The bias of an estimator $\hat \theta_n$ of a parameter $\theta^0$ is defined as

$$B(\hat \theta_n) = E(\hat \theta_n) - \theta^0,$$

where the $n$ subscript indicates that the estimator is a function of the sample size. It follows that the distribution of the estimator is a function of the sample size, meaning that, in general, for each different $n$ the estimator will have a different distribution (maybe only slightly so), which will have a different expected value and so a different bias.

It may not be feasible to obtain information as to how the bias changes (monotonically? non-monotonically?).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.