4
$\begingroup$

I read the following in Machine Learning: A Probabilistic Perspective:

enter image description here

How can a uniform prior move the posterior mean? Isn't a uniform distribution supposed to not bias the result? Are there any other examples where this happens?

Note: In my original question I had misread the paragraph. I fixed the question accordingly.

$\endgroup$
5
$\begingroup$

The uniform prior does not move the mode. The mode of the posterior is equal to the mle in this case. The comparison is made between the posterior expectation, or mean value, and the mle.

From a pragmatic perspective, the argument is a bit silly, as the two values differ by an order of $\frac {2}{N_1+N_0 } $ anyway. So the only time they are different is when you have a small sample size - in which case niether falls outside the "region of uncertainty" associated with the other.

$\endgroup$
  • $\begingroup$ Thanks - you are right. I corrected the question. when you said "the posterior is equal to the MLE in this case" , are there any distributions where the posterior mode is different from the MLE when using a uniform prior? $\endgroup$ – Amelio Vazquez-Reina Mar 16 '14 at 16:01
  • 1
    $\begingroup$ user023472 - posterior $\propto$ likelihood $\times$ prior. If prior is uniform, posterior $\propto$ likelihood. In that case, they necessarily share a mode. (For them not to, the previous argument has to break down somewhere...) $\endgroup$ – Glen_b Mar 16 '14 at 23:05
  • $\begingroup$ This is due to the lack of invariance of the uniform prior. For example, The posterior mode for the odds $\gamma=\frac {\theta}{1-\theta} $ is not equal to the mle of the odds, under a uniform prior for $\theta $ $\endgroup$ – probabilityislogic Mar 17 '14 at 0:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.