What is the mathematical difference between using a un-informative prior and a frequentist approach?

Un-informative priors are preferred in instances where bias is not acceptable (ie. courtrooms, etc.)

However, it seems to me that it would just make sense to use a frequentist approach instead. Why does the Bayesian approach even have a non-informative prior?

Thanks!

• In many situations, there is no such thing as a non-informative prior, because it depends on how the state space is parameterized. So exactly which "non-informative prior" would you hold out as being equivalent to a non-Bayesian analysis? – whuber Sep 8 '14 at 15:14

Even with a non-informative prior, Bayesian inference is different from frequentist approaches. For example, consider estimating the probability $\theta$ that a coin will turn up heads. Take a uniform prior on $\theta$. If we observe a single flip, and it is heads, the Bayesian predictive probability that the next flip will be heads is 2/3. A maximum-likelihood approach would say the probability is 1. If you want the derivation of this result, read Bayesian inference, entropy, and the multinomial distribution.
• I would like to point out that "Frequentist statistics" comprises much more than ML. Indeed, the $2/3$ estimate is a valid Frequentist estimator (it is provably admissible, because it is a Bayes estimator!). Thus this contrast of estimators does not seem like it really illustrates any difference at all between the philosophies. – whuber Sep 8 '14 at 18:38
• Agree with @whuber and I think it is better to think of estimators/algorithms as being paradigm free with the derivation of the estimator being motivated by such-and-such paradigm and the estimator having such-and-such properties under such-and-such paradigm. I'd also like to add that from a frequentist perspective the minimax estimator of $\theta$ in your example is $\frac 3 4$ which agrees with the Jefferys prior. – guy Sep 8 '14 at 19:23