9
$\begingroup$

I am studying different point estimate methods and read that when using MAP vs ML estimates, when we use a "uniform prior", the estimates are identical. Can somebody explain what a "uniform" prior is and give some (simple) examples of when the MAP and ML estimators would be the same?

$\endgroup$
2
10
$\begingroup$

It is a uniform distribution (either continuous or discrete).

See also

http://en.wikipedia.org/wiki/Point_estimation#Bayesian_point-estimation

and

http://en.wikipedia.org/wiki/Maximum_a_posteriori_estimation#Description

If you use a uniform prior on a set that contains the MLE, then MAP=MLE always. The reason for this is that under this prior structure, the posterior distribution and the likelihood are proportional.

$\endgroup$
5
  • 2
    $\begingroup$ This is a good answer in my opinion. It might be worth adding that the reason that the posterior distribution and the likelihood are proportional is that the posterior distribution is itself proportional to the product of the likelihood and the prior. When the prior takes the same value everywhere, as in the uniform distribution, then the posterior distribution is simply proportional to the likelihood. $\endgroup$ – TooTone Jul 14 '13 at 11:37
  • 1
    $\begingroup$ @TooTone I would also add a point about the improperness. $\endgroup$ – Stéphane Laurent Jul 14 '13 at 16:54
  • $\begingroup$ Uniform prior can be viewed as giving a user-set or an equal probability for every class you are trying to predict. For example, if we have two class problem and the distribution for positive examples is 10% (i.e. a prior probability of 0.1), we can set the uniform prior for the positive cases to be 0.5 in order to overcome the imbalance effect of the original distribution. $\endgroup$ – soufanom Jul 17 '13 at 9:17
  • 2
    $\begingroup$ On remark, under uniform prior the MAP and ML collide only if the uniform prior is all over the valid values of the parameter. Namely if the parameter is continuous and the prior is uniform only at [0, 1] it won't hold. $\endgroup$ – Royi Jun 6 '15 at 12:42
  • $\begingroup$ @Drazick: good remark. It is actually "worse" than that, namely the (value of the) MAP depends on the choice of the dominating measure, as explained in this paper of Druihlet and Marin. $\endgroup$ – Xi'an Oct 21 '15 at 20:17
1
$\begingroup$

MLE is the estimate of occurrence of given event given a parameter, whereas MAP is estimate of a parameter given an event. When we use Bayes theorem further while estimating MAP it boils down to $P(D|\theta)P(\theta)$ where $P(\theta)$ is the only additional term with respect to MLE. The mean and variance estimate of MAP will be same as mean and variance estimate of MLE as Prior is remaining the same every time and is not changing at all. Thus it only acts as a constant and thus plays no role in affecting the value of mean and variance.

$\endgroup$
5
  • $\begingroup$ (-1) The maximum likelihood estimate (of a parameter) is an estimate of a parameter, not 'the estimate of occurrence of given event'. The remainder of the answer is somewhat confused/confusing, too, for example it is unclear what the 'mean and variance' refer to. $\endgroup$ – Juho Kokkala Mar 24 '16 at 15:14
  • $\begingroup$ @Tim , Can you provide a proof(or outline) that shows The mean and variance estimate of MAP will be same as mean and variance estimate of MLE ? Thank you $\endgroup$ – curious_dan Mar 30 '19 at 21:56
  • $\begingroup$ @curious_dan Bayes theorem is $p(\theta|X) \propto p(X|\theta) p(\theta)$, if $p(\theta) \propto 1$ is uniform, then it reduces to $p(\theta|X) \propto p(X|\theta) \times 1$, so you are only maximizing the likelihood, so it is the same as MLE. $\endgroup$ – Tim Mar 30 '19 at 21:59
  • $\begingroup$ thank you, @Tim --- I can see why this is true for the maximum/expected value, but it's not clear to me the variance will be the same $\endgroup$ – curious_dan Mar 30 '19 at 22:06
  • $\begingroup$ @curious_dan variance of what? This applies to whatever parameter you estimate. $\endgroup$ – Tim Mar 30 '19 at 22:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.