# About calculating log-likelihood with zeroes

I would like to use the maximum log-likelihood method to find which continuous uniform distribution with the parameters $a$ and $b$ fits best to some observed data values $(x_{0}, \dots, x_{n})$.

I guess the best answer is always $[\min(x), \max(x)]$ but I am using this for the purpose of implementing the relevant algorithms.

The log-likelihood is equal to $$\mathcal{L} = \sum_{i=1}^{n} \log f(x_{i} | a, b).$$

Interestingly, there is a problem with observations $x_{i} < a$ and $x_{i} > b$ (where the density is zero) since the logarithm is undefined at zero.

What's the preferred way to correct for this? I first though to calculate $\log(1 + x_{i})$ (since the optimization algorithm does not care about the plus one) but then I started to think whether some other alternative would be better. Is there any best practice regarding this?

• Sorry there was a mistake in the text. It is now corrected. The problem appears with $x_{i} < a$ and $x_{i} > b$ where the density is zero.
– mmh
Oct 27, 2015 at 14:21
• These are impossible values so the likelihood is zero and the log likelihood minus infinity. Oct 27, 2015 at 14:22
• Well, if there are some $x_i<a$ or $x_i>b$, then you know that your $[a,b]$ can't be right. Your likelihood for such $a,b$ must be zero. Just discard them. Oct 27, 2015 at 14:22
• Commenters weren't suggesting you "delete data". They were suggesting the parameter values should be discarded; they couldn't produce the data you have. Oct 27, 2015 at 15:48
• The problem is universal to the blind use of black-box optimization algorithms, to be sure. It is important to code a function that can be computed for all values satisfying the constraints. For many algorithms, it is important that the function not fail for values that do not satisfy the constraints, too.
– whuber
Oct 27, 2015 at 18:01

The problem is that the objective function of your optimization problem is not defined for all $a$ and $b$. Let's begin by taking a closer look at $f$: $$f(x | a, b) = \begin{cases} \hfill \frac{1}{b - a} \hfill & \text{ if a \leq x \leq b} \\ \hfill 0 \hfill & \text{otherwise} \\ \end{cases}$$ Now let's formulate the optimization problem: $$\underset{a,b}{\text{max}} \sum \log f(x_i | a, b) \\$$ Given the definition of $f$ above, it should be clear that the objective function is only defined when $a \leq x_i \leq b\ \ \forall i$. You could address this problem by maximizing the likelihood instead of the log-likelihood, like so: $$\underset{a,b}{\text{max}} \prod f(x_i | a, b) \\$$ But now you'll have a discontinuity at $a = \min x_i$ and $b = \max x_i$. Most optimization routines are going to have issues with this. So you need to put bounds on your problem. If you properly bound your problem, you can maximize either the likelihood or the log-likelihood: \begin{equation*} \begin{aligned} & \underset{a, b}{\text{max}} & & \sum \log f(x_i | a, b) \\ & \text{subject to} & & a \leq \min x_i, \\ &&& b \geq \max x_i \end{aligned} \end{equation*}