# Bias Correction for Estimator with known bias

To be brief, my question is what could we do when we have an estimator with a known bias.

I want to estimate the parameter $\theta$ in a distribution model.

I select an estimator $\hat\theta$ (e.g. maximum likelihood method), unfortunately according to the training data $x_i, \theta_i$, I find this estimator is biased $E(\hat\theta\mid\theta) = a \theta$, where a is a constant, e.g. $a=0.9$.

To correct this bias, I’d like to use $\hat\theta/a$ as a new unbiased estimator, does this make sense? Is there any terminology for this reduction?

And what if we have $E(\hat\theta\mid\theta) = \theta + a$, where a is constant, then make a correction $\hat\theta-a$?

• Your two expressions for the bias differ: you seem to be assuming $\theta$ will be close to unity. How did you estimate bias with training data only?
– whuber
Commented Aug 12, 2016 at 1:28
• @whuber I've $x_i, \theta_i$ and thus $\hat\theta_i = \hat\theta(x_i)$. Then I can build the probability distribution $p(\hat\theta\mid\theta)$ from the $\hat\theta_i, \theta_i$ pairs. As far as I see, $E(\hat\theta\mid\theta)-\theta$ is the bias of $\hat\theta$. Commented Aug 12, 2016 at 13:41

You may try the method of target estimation to reduce bias, if you have the expectation function estimated either computationally or through closed form expressions or closed form von-mises expansion based approximations. Under certain conditions the bias and/MSE of the target estimator would be better than your initial estimator. Target Estimation: https://projecteuclid.org/download/pdf_1/euclid.aos/1018031269 , Annals of Statistics. There are some other papers on this as well. The target estimator is based on the inverse of the Expectation of a biased estimator.

• Very interesting, thanks for the information. I shall check it later. Commented Oct 6, 2017 at 12:33