What is the mathematical difference between using a un-informative prior and a frequentist approach?

Un-informative priors are preferred in instances where bias is not acceptable (ie. courtrooms, etc.)

However, it seems to me that it would just make sense to use a frequentist approach instead. Why does the Bayesian approach even have a non-informative prior?

Thanks!

• In many situations, there is no such thing as a non-informative prior, because it depends on how the state space is parameterized. So exactly which "non-informative prior" would you hold out as being equivalent to a non-Bayesian analysis?
– whuber
Sep 8 '14 at 15:14

Even with a non-informative prior, Bayesian inference is different from frequentist approaches. For example, consider estimating the probability $\theta$ that a coin will turn up heads. Take a uniform prior on $\theta$. If we observe a single flip, and it is heads, the Bayesian predictive probability that the next flip will be heads is 2/3. A maximum-likelihood approach would say the probability is 1. If you want the derivation of this result, read Bayesian inference, entropy, and the multinomial distribution.

I have written several papers on exactly this topic. If you want more examples, check out: Pathologies of Orthodox Statistics, Inferring a Gaussian distribution and Bayesian inference of a uniform distribution.

• I would like to point out that "Frequentist statistics" comprises much more than ML. Indeed, the $2/3$ estimate is a valid Frequentist estimator (it is provably admissible, because it is a Bayes estimator!). Thus this contrast of estimators does not seem like it really illustrates any difference at all between the philosophies.
– whuber
Sep 8 '14 at 18:38
• Agree with @whuber and I think it is better to think of estimators/algorithms as being paradigm free with the derivation of the estimator being motivated by such-and-such paradigm and the estimator having such-and-such properties under such-and-such paradigm. I'd also like to add that from a frequentist perspective the minimax estimator of $\theta$ in your example is $\frac 3 4$ which agrees with the Jefferys prior.
– guy
Sep 8 '14 at 19:23

It's for methodological purists who cannot bear to use boring 'ol frequentist stats with all their "horrible" inconsistencies (forget the fact that uninformative priors are often improper!).

Seriously, though: An uninformed Bayesian posterior distribution will look an awful lot like a normalized likelihood function, whereas a frequentist would report the usual confidence interval. Since frequentist inference does not obey the likelihood "principle", the two answers may be quite different.

• Could you add some examples? Sep 8 '14 at 7:10