I am studying different point estimate methods and read that when using MAP vs ML estimates, when we use a "uniform prior", the estimates are identical. Can somebody explain what a "uniform" prior is and give some (simple) examples of when the MAP and ML estimators would be the same?
$\begingroup$
$\endgroup$
2
-
5$\begingroup$ @AndreSilva MAP = Maximum a posteriori - the mode of the posterior $\endgroup$– Glen_bJul 14, 2013 at 3:55
-
$\begingroup$ Have a look here: math.stackexchange.com/questions/1327752/… $\endgroup$– RoyiJun 16, 2015 at 17:14
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
5
It is a uniform distribution (either continuous or discrete).
See also
http://en.wikipedia.org/wiki/Point_estimation#Bayesian_point-estimation
and
http://en.wikipedia.org/wiki/Maximum_a_posteriori_estimation#Description
If you use a uniform prior on a set that contains the MLE, then MAP=MLE always. The reason for this is that under this prior structure, the posterior distribution and the likelihood are proportional.
-
2$\begingroup$ This is a good answer in my opinion. It might be worth adding that the reason that the posterior distribution and the likelihood are proportional is that the posterior distribution is itself proportional to the product of the likelihood and the prior. When the prior takes the same value everywhere, as in the uniform distribution, then the posterior distribution is simply proportional to the likelihood. $\endgroup$– TooToneJul 14, 2013 at 11:37
-
2$\begingroup$ @TooTone I would also add a point about the improperness. $\endgroup$ Jul 14, 2013 at 16:54
-
$\begingroup$ Uniform prior can be viewed as giving a user-set or an equal probability for every class you are trying to predict. For example, if we have two class problem and the distribution for positive examples is 10% (i.e. a prior probability of 0.1), we can set the uniform prior for the positive cases to be 0.5 in order to overcome the imbalance effect of the original distribution. $\endgroup$– soufanomJul 17, 2013 at 9:17
-
3$\begingroup$ On remark, under uniform prior the MAP and ML collide only if the uniform prior is all over the valid values of the parameter. Namely if the parameter is continuous and the prior is uniform only at [0, 1] it won't hold. $\endgroup$– RoyiJun 6, 2015 at 12:42
-
1$\begingroup$ @Drazick: good remark. It is actually "worse" than that, namely the (value of the) MAP depends on the choice of the dominating measure, as explained in this paper of Druihlet and Marin. $\endgroup$– Xi'anOct 21, 2015 at 20:17
$\begingroup$
$\endgroup$
5
MLE is the estimate of occurrence of given event given a parameter, whereas MAP is estimate of a parameter given an event. When we use Bayes theorem further while estimating MAP it boils down to $P(D|\theta)P(\theta)$ where $P(\theta)$ is the only additional term with respect to MLE. The mean and variance estimate of MAP will be same as mean and variance estimate of MLE as Prior is remaining the same every time and is not changing at all. Thus it only acts as a constant and thus plays no role in affecting the value of mean and variance.
-
$\begingroup$ (-1) The maximum likelihood estimate (of a parameter) is an estimate of a parameter, not 'the estimate of occurrence of given event'. The remainder of the answer is somewhat confused/confusing, too, for example it is unclear what the 'mean and variance' refer to. $\endgroup$ Mar 24, 2016 at 15:14
-
$\begingroup$ @Tim , Can you provide a proof(or outline) that shows
The mean and variance estimate of MAP will be same as mean and variance estimate of MLE? Thank you $\endgroup$ Mar 30, 2019 at 21:56 -
$\begingroup$ @curious_dan Bayes theorem is $p(\theta|X) \propto p(X|\theta) p(\theta)$, if $p(\theta) \propto 1$ is uniform, then it reduces to $p(\theta|X) \propto p(X|\theta) \times 1$, so you are only maximizing the likelihood, so it is the same as MLE. $\endgroup$– Tim ♦Mar 30, 2019 at 21:59
-
$\begingroup$ thank you, @Tim --- I can see why this is true for the maximum/expected value, but it's not clear to me the variance will be the same $\endgroup$ Mar 30, 2019 at 22:06
-
$\begingroup$ @curious_dan variance of what? This applies to whatever parameter you estimate. $\endgroup$– Tim ♦Mar 30, 2019 at 22:22