Can the likelihood function in MLE be equal to zero?

Question

We have likelihood function

$$ L\left(\mathbf{\Theta} \middle| x_1, \ldots, x_n \right) = \prod^n_{i=1} f\left( x_i \middle| \mathbf{\Theta} \right) $$

and a score function

$$ V\left(\mathbf{\Theta} \middle| x_1, \ldots, x_n \right) = \nabla_\Theta \sum^n_{i=1} \ln{f\left(x_i, \mathbf{\Theta} \right)}, $$

where $x_i$ are observations and $\mathbf{\Theta}$ is the vector of parameters.

If we consider the maximum likelihood estimator $\mathbf{\Theta^{\ast}}$, it's the extreme of $L$, so

$$ V\left(\mathbf{\Theta^{\ast}}\right) = 0, $$

but we need $L\left(\mathbf{\Theta^{\ast}}\middle| x_1, \ldots, x_n \right) \neq 0$.

So, is this inequality always true or is there a situation, where the likelihood with the MLE could be equal to zero?

Answer 1

If you have a very inadequate model such that at least one discretely (or continuously) distributed observation has zero probability (or probability density) for any parameter value $\theta\in\Theta$, that is, you essentially observe something that is impossible under that model, then yes, your maximum likelihood would be zero.

Answer 2

If you observe a sample that has zero probability density under every possible parameter value then the likelihood function is zero over the parameter space. In this case every parameter value is a legitimate value of the MLE (i.e., the MLE is the whole parameter space) and the likelihood is zero at every point that is an MLE. That situation is unusual (and usually entails a misspecified model). The more usual situation is when the likelihood function is positive for at least one parameter value in the parameter space, in which case the likelihood at any MLE point must be positive (proof below).

---

Theorem: Consider a likelihood function $L_\mathbf{x}: \Theta \rightarrow \mathbb{R}$ where we have:

$$L_\mathbf{x}(\theta) > 0 \quad \text{for some } \theta \in \Theta.$$

Any point $\hat{\theta} \in \Theta$ that is an MLE of this likelihood function must satisfy $L_\mathbf{x}(\hat{\theta}) > 0$.

Proof: We proceed using a proof by contradiction. Suppose, contrary to the theorem that there is a point $\hat{\theta}$ that is an MLE and has $L_\mathbf{x}(\hat{\theta}) \leqslant 0$. Since this point is an MLE it must satisfy:

$$L_\mathbf{x}(\hat{\theta}) \geqslant L_\mathbf{x}(\theta) \quad \text{for all } \theta \in \Theta.$$

This implies that $L_\mathbf{x}(\theta) \leqslant L_\mathbf{x}(\hat{\theta}) \leqslant 0$ for all $\theta \in \Theta$ which contradicts the condition on the likelihood in the theorem. By contradiction, this completes the proof. $\blacksquare$

Answer 3

When you define the MLE as the unique point where the likelihood function has a maximum, and if the parameter space consists of multiple points then the likelihood in the MLE must be non-zero.

The reason is that the likelihood must be non-negative and thus zero is the minimum value of the entire range. If the MLE has zero likelihood then this leads to a contradiction. This is because: if the extremum value equals the minimum value of the entire range then there can not be a unique extremum point. Every point will have the same value.

Answer 4

To add a concrete example to the answer by @Jarle Tufto: If $X_1, X_2, \dotsc, X_n$ are iid $\mathcal{U}(0, \theta)$, then any $\theta$ lesser than the maximum observed value give a zero likelihood, since under this model all observations must be lesser (or equal) to $\theta$.