# What causes exponential distribution to have biased and non-biased ML-estimator?

What causes exponential distribution to have biased and non-biased ML-estimator?

$$f(x;\theta)=\theta \exp(-\theta x)$$

has biased estimator.

$$f(x;\theta)=\frac{1}{\beta} \exp(-x/\beta)$$

has unbiased estimator.

But what causes this?

• If the variance is positive, one being unbiased implies the other is biased. There's an informal argument here Nov 21, 2018 at 12:13

1. The MLE of the transform $$T(\theta)$$ is the transform of the MLE $$T(\hat{\theta})$$, i.e., the MLE is equivariant by any one-to-one transform;
2. Unbiasedness does not survive by non-linear transforms, i.e., if $$\mathbb{E}_\theta[\hat{\theta}]=\theta$$ then $$\mathbb{E}_\theta[T(\hat{\theta})]\ne T(\theta)$$
For exponential families, there exists one "mean parameterisation" for which the MLE is unbiased, namely if the density writes $$f(x|\theta)=h(x)\exp\{\theta\cdot S(x)-\tau(\theta)\}$$ then$$\mathbb{E}_\theta[S(X)]=\nabla\tau(\theta)$$ and the MLE $$\hat{\theta}$$ satisfies$$S(X)-\nabla\tau(\hat{\theta})=0$$which implies that $$S(X)$$ is the MLE of its expectation, $$\nabla\tau(\theta)$$, thus that$$\mathbb{E}_\theta[\widehat{\nabla\tau(\theta)}]=\nabla\tau(\theta)$$is unbiased.
• Okay, however, any way to tie this to the example I gave? I don't understand how making $\theta$ to $\frac{1}{\beta}$ would apply to what you're writing. Nov 21, 2018 at 11:26
• Try to fit the exponential distributions within the exponential families framework, i.e., identify $h$, $S$, and $\tau$. Nov 21, 2018 at 11:53
• @mavavilj Let $T(x) = \frac{1}{x}$. Property (1) then gives your maximum likelihood estimator for $\frac{1}{\theta}$. Property (2) says that estimator for $\frac{1}{\theta}$ will be biased. Nov 21, 2018 at 17:03