3
$\begingroup$

I would like to use the maximum log-likelihood method to find which continuous uniform distribution with the parameters $a$ and $b$ fits best to some observed data values $(x_{0}, \dots, x_{n})$.

I guess the best answer is always $[\min(x), \max(x)]$ but I am using this for the purpose of implementing the relevant algorithms.

The log-likelihood is equal to $$\mathcal{L} = \sum_{i=1}^{n} \log f(x_{i} | a, b).$$

Interestingly, there is a problem with observations $x_{i} < a$ and $x_{i} > b$ (where the density is zero) since the logarithm is undefined at zero.

What's the preferred way to correct for this? I first though to calculate $\log(1 + x_{i})$ (since the optimization algorithm does not care about the plus one) but then I started to think whether some other alternative would be better. Is there any best practice regarding this?

Improve this question
$\endgroup$
7
  • $\begingroup$ Sorry there was a mistake in the text. It is now corrected. The problem appears with $x_{i} < a$ and $x_{i} > b$ where the density is zero. $\endgroup$
    – mmh
    Commented Oct 27, 2015 at 14:21
  • 5
    $\begingroup$ These are impossible values so the likelihood is zero and the log likelihood minus infinity. $\endgroup$
    – Xi'an
    Commented Oct 27, 2015 at 14:22
  • 6
    $\begingroup$ Well, if there are some $x_i<a$ or $x_i>b$, then you know that your $[a,b]$ can't be right. Your likelihood for such $a,b$ must be zero. Just discard them. $\endgroup$
    – Stephan Kolassa
    Commented Oct 27, 2015 at 14:22
  • 4
    $\begingroup$ Commenters weren't suggesting you "delete data". They were suggesting the parameter values should be discarded; they couldn't produce the data you have. $\endgroup$
    – Glen_b
    Commented Oct 27, 2015 at 15:48
  • 4
    $\begingroup$ The problem is universal to the blind use of black-box optimization algorithms, to be sure. It is important to code a function that can be computed for all values satisfying the constraints. For many algorithms, it is important that the function not fail for values that do not satisfy the constraints, too. $\endgroup$
    – whuber ♦
    Commented Oct 27, 2015 at 18:01

1 Answer 1

Reset to default
4
$\begingroup$

The problem is that the objective function of your optimization problem is not defined for all $a$ and $b$. Let's begin by taking a closer look at $f$: $$ f(x | a, b) = \begin{cases} \hfill \frac{1}{b - a} \hfill & \text{ if $a \leq x \leq b$} \\ \hfill 0 \hfill & \text{otherwise} \\ \end{cases} $$ Now let's formulate the optimization problem: $$ \underset{a,b}{\text{max}} \sum \log f(x_i | a, b) \\ $$ Given the definition of $f$ above, it should be clear that the objective function is only defined when $a \leq x_i \leq b\ \ \forall i$. You could address this problem by maximizing the likelihood instead of the log-likelihood, like so: $$ \underset{a,b}{\text{max}} \prod f(x_i | a, b) \\ $$ But now you'll have a discontinuity at $a = \min x_i$ and $b = \max x_i$. Most optimization routines are going to have issues with this. So you need to put bounds on your problem. If you properly bound your problem, you can maximize either the likelihood or the log-likelihood: $$ \begin{equation*} \begin{aligned} & \underset{a, b}{\text{max}} & & \sum \log f(x_i | a, b) \\ & \text{subject to} & & a \leq \min x_i, \\ &&& b \geq \max x_i \end{aligned} \end{equation*} $$

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ Once you reach this stage, an optimisation routine sounds superfluous to solve the maximisation problem. $\endgroup$
    – Xi'an
    Commented Oct 28, 2015 at 10:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.