1
$\begingroup$

Consider $N$ observed data points $x_i$ ($i=1,..,N$), and a likelihood that depends on $p$ parameters: $f(x_i|\theta_n)$ ($n=1,..p$). From Bayes' theorem

$$p(\theta_n|x_i) = \frac{f(x_i|\theta_n)g(\theta_n)}{h(x_i)}$$

I apply an MCMC sampler and obtain as a result the (sampled) posterior $p(\theta_n|x_i)$, and a (sampled) probability distribution for each $\theta_n$ individually (from where I can obtain for example the mean and the median for each $\theta_n$).

By definition, the maximum a posteriori (MAP) is obtained as:

$$\hat{\theta}_{\mathrm{MAP}}(x_i) = \underset{\theta_n}{\operatorname{arg\,max}} \ p(\theta_n|x_i) $$

This is where I get lost. Are the MAP estimates of each $\theta_n$:

  1. the values associated to the maximum value of the (sampled) full posterior $p(\theta_n|x_i)$?, or
  2. the values obtained finding the maximum (ie: the mode) in the (sampled) distribution of each single $\theta_n$ parameter?

To illustrate what I mean, here's the result of an MCMC run showing the sampled distribution of a single $\theta_n$ parameter:

enter image description here

You can see that the mode (cyan line) of the distribution for this parameter is very different from the MAP (red line) value that I obtain from the maximum value of the sampled posterior $p(\theta_n|x_i)$.

$\endgroup$
2
$\begingroup$

There is a unique* maximum a posteriori estimate (MAP) associated with the parameter vector $\theta=(\theta_1,\ldots,\theta_p)$. The MAP is the value that maximizes the posterior distribution, i.e.

$$ \hat{\theta}_{MAP} = \text{argmax}_\theta p(\theta|x) = \text{argmax}_\theta f(x|\theta)g(\theta) $$

where $f(x|\theta)$ is the joint density/mass function of all of your observations $x=(x_1,\ldots,x_N)$ conditional on the parameter vector $\theta$ and $g(\theta)$ is the prior density for $\theta$.

You have not provided enough information in the problem to understand how to construct $f(x|\theta)$ given the individual $f(x_i|\theta_n)$. This notation actually doesn't make much sense as there doesn't seem to be a correspondence between $i$ and $n$.

*Technically the MAP does not need to be unique, but it often is.

$\endgroup$
  • $\begingroup$ There is no correspondence between $i$ and $n$. The former runs through all my observed data, and the latter through the parameters of my model. Perhaps I over-complicated the notation? $\endgroup$ – Gabriel Apr 13 '18 at 21:54
  • $\begingroup$ Why does the form of the likelihood ($f(x|\theta)$) matter? $\endgroup$ – Gabriel Apr 13 '18 at 21:57
  • $\begingroup$ Am I understanding correctly then that the MAP is obtained maximizing the full posterior distribution ant not the distribution of each parameter separately? $\endgroup$ – Gabriel Apr 16 '18 at 15:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.