4
$\begingroup$

What causes exponential distribution to have biased and non-biased ML-estimator?

$f(x;\theta)=\theta \exp(-\theta x)$

has biased estimator.

$f(x;\theta)=\frac{1}{\beta} \exp(-x/\beta)$

has unbiased estimator.

But what causes this?

$\endgroup$
1
  • 1
    $\begingroup$ If the variance is positive, one being unbiased implies the other is biased. There's an informal argument here $\endgroup$
    – Glen_b
    Nov 21, 2018 at 12:13

1 Answer 1

3
$\begingroup$

The simple explanation is that

  1. The MLE of the transform $T(\theta)$ is the transform of the MLE $T(\hat{\theta})$, i.e., the MLE is equivariant by any one-to-one transform;
  2. Unbiasedness does not survive by non-linear transforms, i.e., if $\mathbb{E}_\theta[\hat{\theta}]=\theta$ then $\mathbb{E}_\theta[T(\hat{\theta})]\ne T(\theta)$

For exponential families, there exists one "mean parameterisation" for which the MLE is unbiased, namely if the density writes $$f(x|\theta)=h(x)\exp\{\theta\cdot S(x)-\tau(\theta)\}$$ then$$\mathbb{E}_\theta[S(X)]=\nabla\tau(\theta)$$ and the MLE $\hat{\theta}$ satisfies$$S(X)-\nabla\tau(\hat{\theta})=0$$which implies that $S(X)$ is the MLE of its expectation, $\nabla\tau(\theta)$, thus that$$\mathbb{E}_\theta[\widehat{\nabla\tau(\theta)}]=\nabla\tau(\theta)$$is unbiased.

$\endgroup$
3
  • $\begingroup$ Okay, however, any way to tie this to the example I gave? I don't understand how making $\theta$ to $\frac{1}{\beta}$ would apply to what you're writing. $\endgroup$
    – mavavilj
    Nov 21, 2018 at 11:26
  • $\begingroup$ Try to fit the exponential distributions within the exponential families framework, i.e., identify $h$, $S$, and $\tau$. $\endgroup$
    – Xi'an
    Nov 21, 2018 at 11:53
  • $\begingroup$ @mavavilj Let $T(x) = \frac{1}{x}$. Property (1) then gives your maximum likelihood estimator for $\frac{1}{\theta}$. Property (2) says that estimator for $\frac{1}{\theta}$ will be biased. $\endgroup$ Nov 21, 2018 at 17:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.