What causes exponential distribution to have biased and non-biased ML-estimator?

$f(x;\theta)=\theta \exp(-\theta x)$

has biased estimator.

$f(x;\theta)=\frac{1}{\beta} \exp(-x/\beta)$

has unbiased estimator.

But what causes this?

  • 1
    $\begingroup$ If the variance is positive, one being unbiased implies the other is biased. There's an informal argument here $\endgroup$ – Glen_b Nov 21 '18 at 12:13

The simple explanation is that

  1. The MLE of the transform $T(\theta)$ is the transform of the MLE $T(\hat{\theta})$, i.e., the MLE is equivariant by any one-to-one transform;
  2. Unbiasedness does not survive by non-linear transforms, i.e., if $\mathbb{E}_\theta[\hat{\theta}]=\theta$ then $\mathbb{E}_\theta[T(\hat{\theta})]\ne T(\theta)$

For exponential families, there exists one "mean parameterisation" for which the MLE is unbiased, namely if the density writes $$f(x|\theta)=h(x)\exp\{\theta\cdot S(x)-\tau(\theta)\}$$ then$$\mathbb{E}_\theta[S(X)]=\nabla\tau(\theta)$$ and the MLE $\hat{\theta}$ satisfies$$S(X)-\nabla\tau(\hat{\theta})=0$$which implies that $S(X)$ is the MLE of its expectation, $\nabla\tau(\theta)$, thus that$$\mathbb{E}_\theta[\widehat{\nabla\tau(\theta)}]=\nabla\tau(\theta)$$is unbiased.

  • $\begingroup$ Okay, however, any way to tie this to the example I gave? I don't understand how making $\theta$ to $\frac{1}{\beta}$ would apply to what you're writing. $\endgroup$ – mavavilj Nov 21 '18 at 11:26
  • $\begingroup$ Try to fit the exponential distributions within the exponential families framework, i.e., identify $h$, $S$, and $\tau$. $\endgroup$ – Xi'an Nov 21 '18 at 11:53
  • $\begingroup$ @mavavilj Let $T(x) = \frac{1}{x}$. Property (1) then gives your maximum likelihood estimator for $\frac{1}{\theta}$. Property (2) says that estimator for $\frac{1}{\theta}$ will be biased. $\endgroup$ – Matthew Gunn Nov 21 '18 at 17:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.