5
$\begingroup$

According to the WP definition of MAP:

In Bayesian statistics, a maximum a posteriori probability (MAP) estimate is an estimate of an unknown quantity, that equals the mode of the posterior distribution.

(emphasis added) which, given a posterior $f(\theta \mid x)$, can be defined as:

$$\hat{\theta}_{\mathrm{MAP}}(x) = \underset{\theta}{\operatorname{arg\,max}} \ f(\theta \mid x) $$

As far as I understand it, the mode of a distribution depends on how I construct its histogram (or KDE). This looks to me to be in contradiction with the above definition, where the MAP is the $\theta$ value found for the maximum of the sampled $f(\theta \mid x)$ and does not depend on anything else.

What am I missing?

$\endgroup$
9
$\begingroup$

In Bayesian statistics, a maximum a posteriori probability (MAP) estimate is an estimate of an unknown quantity, that equals the mode of the posterior distribution.

This is correct.

As far as I understand it, the mode of a distribution depends on how I construct its histogram (or KDE). This looks to me to be in contradiction with the above definition, where the MAP is the maximum θ value found for the sampled f(θ∣x) and does not depend on anything else.

You are confusing theoretical quantities with random/sampling based ones. The posterior is defined by the likelihood and prior. That is $f(θ∣x) = \frac{L(\theta)\pi(\theta)}{\int_{\Theta} L(\theta')\pi(\theta') d \theta' }$ where $L(\theta)$, $\pi(\theta)$ are the likelihood and prior, respectively. This is a unique function in $\theta$.

In practice it is common to sample from this distribution, and if that's the case, then you're introducing random error. Usually when this is done, it is the only option because, say, the normalizing constant will not be available. Popular strategies obtain samples from $f(\theta|x)$ and only require that the user is able to evaluate the un-normalized posterior $L(\theta)\pi(\theta)$. Even though this is only an approximation because it is indeed based on random draws, there are results that guarantee the convergence of your estimators, so the error is tolerable as long as you run your simulations for long enough.

$\endgroup$
  • $\begingroup$ So, if I'm understanding what you said: the theoretical mode of $f(\theta\mid x)$ is always equal to its maximum value (MAP), but since I'm using a sampled $f(\theta\mid x)$ the mode and the MAP will not necessarily match? $\endgroup$ – Gabriel Oct 16 '17 at 18:19
  • 1
    $\begingroup$ @Gabriel correct $\endgroup$ – Taylor Oct 16 '17 at 18:20
-1
$\begingroup$

The mode of distribution is either the value with more probability (for discrete random variable) or the point with highest probability density value (for continuous random variables). So yes, the mode of the a posteriori distribution is the MAP estimate.

Note that, when constructing a histogram, you don't know the real distribution, which means the "mode" you find is just an estimate of what interval has the most probability associated.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.