To be brief, my question is what could we do when we have an estimator with a known bias.

I want to estimate the parameter $\theta$ in a distribution model.

I select an estimator $\hat\theta$ (e.g. maximum likelihood method), unfortunately according to the training data $x_i, \theta_i$, I find this estimator is biased $E(\hat\theta\mid\theta) = a \theta$, where a is a constant, e.g. $a=0.9$.

To correct this bias, I’d like to use $\hat\theta/a$ as a new unbiased estimator, does this make sense? Is there any terminology for this reduction?

And what if we have $E(\hat\theta\mid\theta) = \theta + a$, where a is constant, then make a correction $\hat\theta-a$?

  • 1
    $\begingroup$ Your two expressions for the bias differ: you seem to be assuming $\theta$ will be close to unity. How did you estimate bias with training data only? $\endgroup$
    – whuber
    Commented Aug 12, 2016 at 1:28
  • $\begingroup$ @whuber I've $x_i, \theta_i$ and thus $\hat\theta_i = \hat\theta(x_i)$. Then I can build the probability distribution $p(\hat\theta\mid\theta)$ from the $\hat\theta_i, \theta_i$ pairs. As far as I see, $E(\hat\theta\mid\theta)-\theta$ is the bias of $\hat\theta$. $\endgroup$ Commented Aug 12, 2016 at 13:41

1 Answer 1


You may try the method of target estimation to reduce bias, if you have the expectation function estimated either computationally or through closed form expressions or closed form von-mises expansion based approximations. Under certain conditions the bias and/MSE of the target estimator would be better than your initial estimator. Target Estimation: https://projecteuclid.org/download/pdf_1/euclid.aos/1018031269 , Annals of Statistics. There are some other papers on this as well. The target estimator is based on the inverse of the Expectation of a biased estimator.

  • $\begingroup$ Very interesting, thanks for the information. I shall check it later. $\endgroup$ Commented Oct 6, 2017 at 12:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.