1
$\begingroup$

Suppose I have the joint distribution:

\begin{align} p(\mathbf{x}) = p(x_1, ... , x_n) \end{align}

The maximum a posteriori (MAP) solution is given by:

\begin{align} \mathbf{x}_{MAP} = \arg \max p(\mathbf{x}) \end{align}

Intuitively I believe that finding the maximums of each marginal should not always correspond to $\mathbf{x}_{MAP}$. That is,

\begin{align} \mathbf{x}_{MAP_1} \neq \max_{x_1}(p(x_1))\\ \mathbf{x}_{MAP_2} \neq \max_{x_2}(p(x_2)) \\ \vdots \\ \mathbf{x}_{MAP_n} \neq \max_{x_n}(p(x_n)) \end{align}

I believe this should be the case, HOWEVER I can't convince myself fully. Every joint distribution I imagine, tends to satisfy this trivially (e.g. the mode of a multi-variate Gaussian, can be found by considering the mode for each of the marginals).

  1. Is there/are there counter-examples to prove this point?
  2. Or perhaps is my intuition wrong? That is, the mode of the marginals always corresponds to the MAP mode.
  3. Are there sources on the internet which explore this issue in depth (since no matter what I Google I cannot find an answer on this)?
  4. Are there any conditions to declare when this is / isn't true?
Improve this question
$\endgroup$
6
  • $\begingroup$ What is the connection between $p(\boldsymbol{x})$ and $p(x_1)$? $\endgroup$
    – tmrlvi
    Commented Aug 3, 2019 at 15:57
  • $\begingroup$ $\mathbf{x}$ is a vector and its components are given by $x_1, ..., x_n$. Let me know if you need further information, since I may have abused notation here. $p(\mathbf{x})$ is shorthand for the joint distribution $\endgroup$
    – tisPrimeTime
    Commented Aug 3, 2019 at 15:58
  • $\begingroup$ I'm asking about the connections between the density of the multivariate distribution and each marginal, since the answer depends mainly on that. For example, What is the connection between $p(\boldsymbol{x})$ and $p(x_1)$? if $p(\boldsymbol{x}) =\prod_{i=1}^{n} p(x_{i})$, then the components of the MAP over all is in fact the MAP in each marginal. Otherwise, we can build a counter example. $\endgroup$
    – tmrlvi
    Commented Aug 3, 2019 at 16:00
  • $\begingroup$ The relationship is arbitrary. Hence why I would love to see some counter examples of why the MAP over the joint doesn't necessarily correspond to the individual maximums across each marginal. :) $\endgroup$
    – tisPrimeTime
    Commented Aug 3, 2019 at 16:02
  • $\begingroup$ I assume you meant $\arg \max$, not $\arg \min$, in your second equation? $\endgroup$
    – jbowman
    Commented Aug 3, 2019 at 20:08

1 Answer 1

Reset to default
1
$\begingroup$

Consider the following discrete distribution:

$$ \boldsymbol{X} = \begin{cases} (1,1) & \text{w.p. } 0.3 \\ (1,0) & \text{w.p. } 0.15 \\ (0,1) & \text{w.p. } 0.275 \\ (0,0) & \text{w.p. } 0.275 \\ \end{cases} $$

Then, $$ \boldsymbol{x}_{\text{MAP}} = (1,1) $$

But $$ \arg\max p(x_1)= 0 \neq \left(\boldsymbol{x}_{\text{MAP}}\right)_{1} $$ Since $P(X_1 =1) = 0.45$ and $P(X_1 =0) = 0.55$

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks! (1) I think the second expression is supposed to be $P(X_1 = 0)$? Also (2) Would you happen to know of any additional example(s) involving continuous distributions? $\endgroup$
    – tisPrimeTime
    Commented Aug 3, 2019 at 16:20
  • $\begingroup$ (1) You are right. I fixed it. (2) I believe such exists, but I'm not sure yet. $\endgroup$
    – tmrlvi
    Commented Aug 3, 2019 at 16:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.