5
$\begingroup$

I am learning Bayesian statistics. I found that this pymc3 introduction sometimes uses MAP to estimate the parameters for the MCMC input (the regression example), but the intro doesn't run MAP for Stochastic Vol and Coal Disaster cases.

I understand that when the mode is not representative or if we have experts specifying explicitly the prior we shouldn't use MAP.

Assuming that we know nothing much about the data, and just draft the hierarchical model and we have no help from experts, should we always run MAP to find out the initial point estimates before doing MCMC? e.g. The stochastic vol and coal disaster examples should we also run MAP?

$\endgroup$
5
  • 3
    $\begingroup$ There is some degree of confusion in the question: finding a MAP or ML estimate of the parameters, whenever manageable, is a good way to initialise an MCMC algorithm because it starts at a value of the parameters that is realistic for the data at hand, hence removes the warm-up steps of the algorithm. $\endgroup$ – Xi'an Mar 21 '18 at 9:27
  • $\begingroup$ Sorry for the confusion. Yes that's also my understanding that MAP finds a good estimate for init parameters. My question is actually asking under what situations (besides the two I mentioned) we shouldn't / couldn't use MAP to find the initial parameters for MCMC. $\endgroup$ – Paul Mar 21 '18 at 9:33
  • 2
    $\begingroup$ The only case against using the MAP, besides the one when the value produced by a MAP algorithm is actually far from the true MAP (!), is when the MAP is located at the tip of a tiny spike in a low probability region (which is not in contradiction with the MAP displaying the largest density value). $\endgroup$ – Xi'an Mar 21 '18 at 9:52
  • 1
    $\begingroup$ @Xi'an Note however that the MAP being in tiny spike is not uncommon - it arises for example in the "standard" parametrization of hierarchical models: see for example arxiv.org/pdf/1312.0906.pdf for further discussion. $\endgroup$ – Martin Modrák Mar 21 '18 at 12:12
  • 1
    $\begingroup$ @MartinModrák: or even more simply in a large dimensional Gaussian - like model, where most of the mass is on the sphere rather than at the mode. $\endgroup$ – Xi'an Mar 21 '18 at 13:53
2
$\begingroup$

MAP is mode of the posterior distribution, as noticed in the comments, it does not have to be a reasonable estimate to consider. Likely, you can see MAP in the tutorials, because they want to show different possible functionalities of their software, rather then the most methodologically sound solution. In some cases it may be reasonable to use MAP as a starting point for sampling, since this enables sampler to start at a reasonable starting point, what should give you the reasonable samples faster, then if started from completely random initialization. Notice however that this does not have to work, for example PyMC3 documentation discourages using MAP as an initialization for NUTS sampler and uses different form of initialization as a default. So definitely this is not a one-size-fit-all solution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.