5
$\begingroup$

We have likelihood function

$$ L\left(\mathbf{\Theta} \middle| x_1, \ldots, x_n \right) = \prod^n_{i=1} f\left( x_i \middle| \mathbf{\Theta} \right) $$

and a score function

$$ V\left(\mathbf{\Theta} \middle| x_1, \ldots, x_n \right) = \nabla_\Theta \sum^n_{i=1} \ln{f\left(x_i, \mathbf{\Theta} \right)}, $$

where $x_i$ are observations and $\mathbf{\Theta}$ is the vector of parameters.

If we consider the maximum likelihood estimator $\mathbf{\Theta^{\ast}}$, it's the extreme of $L$, so

$$ V\left(\mathbf{\Theta^{\ast}}\right) = 0, $$

but we need $L\left(\mathbf{\Theta^{\ast}}\middle| x_1, \ldots, x_n \right) \neq 0$.

So, is this inequality always true or is there a situation, where the likelihood with the MLE could be equal to zero?

$\endgroup$

4 Answers 4

4
$\begingroup$

If you have a very inadequate model such that at least one discretely (or continuously) distributed observation has zero probability (or probability density) for any parameter value $\theta\in\Theta$, that is, you essentially observe something that is impossible under that model, then yes, your maximum likelihood would be zero.

$\endgroup$
1
  • 1
    $\begingroup$ But would you have a maximum likelihood estimate for such a case? If the observation has zero probability for any possible parameter value, what is the MLE? I would say that: there is an MLE only if there is a non-zero positive likelihood and therefore the likelihood function in the MLE must be non-zero. $\endgroup$ Commented Jan 29, 2021 at 10:12
3
$\begingroup$

If you observe a sample that has zero probability density under every possible parameter value then the likelihood function is zero over the parameter space. In this case every parameter value is a legitimate value of the MLE (i.e., the MLE is the whole parameter space) and the likelihood is zero at every point that is an MLE. That situation is unusual (and usually entails a misspecified model). The more usual situation is when the likelihood function is positive for at least one parameter value in the parameter space, in which case the likelihood at any MLE point must be positive (proof below).


Theorem: Consider a likelihood function $L_\mathbf{x}: \Theta \rightarrow \mathbb{R}$ where we have:

$$L_\mathbf{x}(\theta) > 0 \quad \text{for some } \theta \in \Theta.$$

Any point $\hat{\theta} \in \Theta$ that is an MLE of this likelihood function must satisfy $L_\mathbf{x}(\hat{\theta}) > 0$.

Proof: We proceed using a proof by contradiction. Suppose, contrary to the theorem that there is a point $\hat{\theta}$ that is an MLE and has $L_\mathbf{x}(\hat{\theta}) \leqslant 0$. Since this point is an MLE it must satisfy:

$$L_\mathbf{x}(\hat{\theta}) \geqslant L_\mathbf{x}(\theta) \quad \text{for all } \theta \in \Theta.$$

This implies that $L_\mathbf{x}(\theta) \leqslant L_\mathbf{x}(\hat{\theta}) \leqslant 0$ for all $\theta \in \Theta$ which contradicts the condition on the likelihood in the theorem. By contradiction, this completes the proof. $\blacksquare$

$\endgroup$
0
1
$\begingroup$

When you define the MLE as the unique point where the likelihood function has a maximum, and if the parameter space consists of multiple points then the likelihood in the MLE must be non-zero.

The reason is that the likelihood must be non-negative and thus zero is the minimum value of the entire range. If the MLE has zero likelihood then this leads to a contradiction. This is because: if the extremum value equals the minimum value of the entire range then there can not be a unique extremum point. Every point will have the same value.

$\endgroup$
0
$\begingroup$

To add a concrete example to the answer by @Jarle Tufto: If $X_1, X_2, \dotsc, X_n$ are iid $\mathcal{U}(0, \theta)$, then any $\theta$ lesser than the maximum observed value give a zero likelihood, since under this model all observations must be lesser (or equal) to $\theta$.

$\endgroup$
2
  • $\begingroup$ This is an example where the likelihood function is zero in the points that are not the MLE. Is there a concrete example where the likelihood function in the MLE is equal to zero? $\endgroup$ Commented Jan 29, 2021 at 10:13
  • $\begingroup$ I misread the Q---A model, such as the lognormal or exponential distribution giving zero probability to negative observations, and a sample containing negative observations ... then the likelihood function will be identically zero. $\endgroup$ Commented Jan 29, 2021 at 14:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.