0
$\begingroup$

Given a N-parameter likelihood and prior, I can obtain the marginalized posterior for each parameter through Bayesian MCMC.

I can also estimate the maximum a posteriori (MAP) of the N-parameter posterior distribution $P(\theta|D)$ (which equals its mode).

I'm having trouble visualizing the fact that, for any given parameter $\theta_i\; (i=1, N)$, its MAP value does not necessarily match its mean value (taken from its marginalized posterior).

Can you give an example of a distribution where this is visualized?

$\endgroup$
2
$\begingroup$

The marginalized posterior of a parameter doesn't have to be symmetric and have its mode at the center, which would be the case most people think of (e.g., the Normal distribution) when envisioning the posterior. In fact, by choosing your prior appropriately, the effect you mention can happen even with a "nice" likelihood like that of the Normal distribution, but my example will be a little different.

Consider the posterior distribution for the mean parameter $\theta$ of an Exponential distribution with a prior distribution ($p$) uniform on $(0,100000)$: $p(\theta) = {1 \over 100000} \space \text{I}_{(0, 100000)}$. Let's assume the true parameter value $\theta = 1$, which renders the upper bound effectively meaningless when calculating the posterior, as it is far larger than any sample will ever indicate $\theta$ might be. The posterior distribution ($p'$) for a sample of size $N$ then becomes proportional to:

$$p'(\theta) \propto {1\over \theta^N}\text{e}^{-\sum{x}/\theta}\text{I}_{(0,10000)}$$

and, for all practical purposes, we can ignore the upper bound on the range of $\theta$. This is an Inverted Gamma distribution with shape parameter $N-1$ and scale parameter $\sum x$, and has a mean of ${\sum x \over N-2}$ but mode (MAP value) $\sum x \over N$.

Since this can happen for the univariate case, you should be able to see that it can also happen for the multivariate case where we have a marginalized posterior instead.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.