12
$\begingroup$

Un-informative priors are preferred in instances where bias is not acceptable (ie. courtrooms, etc.)

However, it seems to me that it would just make sense to use a frequentist approach instead. Why does the Bayesian approach even have a non-informative prior?

Thanks!

Improve this question
$\endgroup$
1
  • 4
    $\begingroup$ In many situations, there is no such thing as a non-informative prior, because it depends on how the state space is parameterized. So exactly which "non-informative prior" would you hold out as being equivalent to a non-Bayesian analysis? $\endgroup$
    – whuber ♦
    Commented Sep 8, 2014 at 15:14

2 Answers 2

Reset to default
3
$\begingroup$

Even with a non-informative prior, Bayesian inference is different from frequentist approaches. For example, consider estimating the probability $\theta$ that a coin will turn up heads. Take a uniform prior on $\theta$. If we observe a single flip, and it is heads, the Bayesian predictive probability that the next flip will be heads is 2/3. A maximum-likelihood approach would say the probability is 1. If you want the derivation of this result, read Bayesian inference, entropy, and the multinomial distribution.

I have written several papers on exactly this topic. If you want more examples, check out: Pathologies of Orthodox Statistics, Inferring a Gaussian distribution and Bayesian inference of a uniform distribution.

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ I would like to point out that "Frequentist statistics" comprises much more than ML. Indeed, the $2/3$ estimate is a valid Frequentist estimator (it is provably admissible, because it is a Bayes estimator!). Thus this contrast of estimators does not seem like it really illustrates any difference at all between the philosophies. $\endgroup$
    – whuber ♦
    Commented Sep 8, 2014 at 18:38
  • 2
    $\begingroup$ Agree with @whuber and I think it is better to think of estimators/algorithms as being paradigm free with the derivation of the estimator being motivated by such-and-such paradigm and the estimator having such-and-such properties under such-and-such paradigm. I'd also like to add that from a frequentist perspective the minimax estimator of $\theta$ in your example is $\frac 3 4$ which agrees with the Jefferys prior. $\endgroup$
    – guy
    Commented Sep 8, 2014 at 19:23
1
$\begingroup$

It's for methodological purists who cannot bear to use boring 'ol frequentist stats with all their "horrible" inconsistencies (forget the fact that uninformative priors are often improper!).

Seriously, though: An uninformed Bayesian posterior distribution will look an awful lot like a normalized likelihood function, whereas a frequentist would report the usual confidence interval. Since frequentist inference does not obey the likelihood "principle", the two answers may be quite different.

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ Could you add some examples? $\endgroup$
    – Yehoshaphat Schellekens
    Commented Sep 8, 2014 at 7:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.