3
$\begingroup$

Let $p$ be a probability distribution and let $\mathcal{D}_1$, $\mathcal{D}_2$ be two sets of observations.

If the likelihood of the parameter for some observations $$ \mathcal{L}(\theta; \mathcal{D}) = p(\mathcal{D} | \theta) $$ has a single mode, is it true that if we have

$$ \mathcal{L}(\theta; \mathcal{D}) = p(\mathcal{D}_1 | \theta) + p(\mathcal{D}_2 | \theta) $$ The likelihood over the parameter $\theta$ has a single mode as well?

If not,

  • Are there some conditions or probability distributions under which this is true?

  • What is usually done to find the MLE?

$\endgroup$
  • $\begingroup$ Is your problem regarding the sum of the random variables or the sum of the likelihoods? It's not at all the same. Why would you add two likelihoods? The sum is not a likelihood. Why would you optimize the sum for θ? $\endgroup$ – Zahava Kor Mar 8 '17 at 20:45
  • $\begingroup$ The addition is due to the fact that the data might be either D_1 or D_2. Both would be consistent with the real observation. $\endgroup$ – alberto Mar 9 '17 at 10:37
1
$\begingroup$

The premise of this question appears to be false -- you don't add likelihoods like this. If the two sets of observations are independent, you would multiply likelihoods (add log-likelihoods).

Nevertheless (assuming it made sense in some context to even consider the question), it is not generally the case that sums of unimodal likelihoods would be unimodal.

It's also not the case that sums of unimodal log-likelihoods would be unimodal (e.g. consider the Cauchy which is often a counterexample).

$\endgroup$
  • 1
    $\begingroup$ Thanks Glen. In my context, it makes sense because it is like ignoring the real value of the observation. Then the wanted MLE is the one that maximizes the likelihood on this union set of possible values (not all possible values in the space, but the subset which is consistent with some other observation) $\endgroup$ – alberto Mar 9 '17 at 10:20
  • 1
    $\begingroup$ However, it makes sense when you switch to log-likelihoods. Which extremes are the same as the corresponding likelihoods. $\endgroup$ – Xi'an Mar 9 '17 at 10:44
  • 1
    $\begingroup$ @Xi'an Given the original question at stats.stackexchange.com/questions/265980/…, adding pmf's (not really likelihoods) makes sense and may lead to a multimodal likelihood/posterior distribution $\endgroup$ – Jarle Tufto Mar 9 '17 at 14:44
6
$\begingroup$

Take the sum of two variance one normals, centred at $\bar{x}_1=0$ and $\bar{x}_2=4$. Then you get the following graph:

enter image description here

$\endgroup$
  • $\begingroup$ And therefore you find the MLE parameter by some optimization method? (assuming we cannot derivate it analytically) $\endgroup$ – alberto Mar 8 '17 at 18:02
  • 2
    $\begingroup$ In this extreme case, there are two equivalent MLEs. In general, you do find the MLE by an optimisation method so I am unsure about the question. MLEs exist even with multimodal likelihoods. $\endgroup$ – Xi'an Mar 8 '17 at 18:04
  • $\begingroup$ Thanks Xi'an. The thing is that I have a very long sum of probabilities (see stats.stackexchange.com/questions/265980/…) and I wanted to be sure that I'm not missing something trivial. $\endgroup$ – alberto Mar 8 '17 at 18:11
  • 1
    $\begingroup$ Sums are not ideal objects to optimise, but they come fairly naturally in statistics. $\endgroup$ – Xi'an Mar 8 '17 at 18:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.