14
$\begingroup$

As a student in physics, I have experienced the "Why I am a Bayesian" lecture perhaps half a dozen times. It is always the same -- the presenter smugly explains how the Bayesian interpretation is superior to the frequentist interpretation allegedly employed by the masses. They mention Bayes rule, marginalization, priors and posteriors.

What is the real story?

Is there a legitimate domain of applicability for frequentist statistics? (Surely in sampling or rolling a die many times it must apply?)

Are there useful probabilistic philosophies beyond "bayesian" and "frequentist"?

$\endgroup$

migrated from physics.stackexchange.com Jun 9 '11 at 18:42

This question came from our site for active researchers, academics and students of physics.

  • 2
    $\begingroup$ This isn't about physics, it's about statistics and therefore it probably belongs on our statistics site. At least, it doesn't belong here. I'll temporarily close it as "too localized" (not because it is, just because I need a close reason) and if the people on stats.SE want it, I'll migrate it there. $\endgroup$ – David Zaslavsky Jun 7 '11 at 16:56
  • 1
    $\begingroup$ I think this is a perfectly reasonable physics question. Experimental physicists are obsessed with bayesian statistics and use them daily. I want to know "what else is out there" and whether it is also useful to an (experimental) physicist. P.S. What's up with this immediate, do-not-pass-go closure? I thought there was "vote to close (N votes needed)". $\endgroup$ – nibot Jun 7 '11 at 19:29
  • 2
    $\begingroup$ @nibot: and theoretical physicists are obsessed with integrals and use them daily. That doesn't mean questions about integration are on-topic for this site. Same with your question; just because it is about something that physicists use doesn't mean it is about physics. P.S. Moderator close votes are binding. There are not that many people here with enough reputation to cast close votes, so it usually falls to the moderators. $\endgroup$ – David Zaslavsky Jun 7 '11 at 20:02
  • 1
    $\begingroup$ @nibot: I agree with David, it's a stats question albeit one of interest to physicists. But also of interest to biologists, psychologists, and a great many other ists. The one-vote close is because David is a moderator (note the "♦"). $\endgroup$ – dmckee Jun 7 '11 at 20:04
  • $\begingroup$ It could also be on topic at the slightly broader math stackexchange. $\endgroup$ – wnoise Jun 7 '11 at 21:01
11
$\begingroup$

The Bayesian interpretation of probability suffices for practical purposes. But even given a Bayesian interpretation of probability, there is more to statistics than probability, because the foundation of statistics is decision theory and decision theory requires not only a class of probability models but also the specification of a optimality criteria for a decision rule. Under Bayes criteria, the optimal decision rules can be obtained through Bayes' rule; but many frequentist methods are justified under minimax and other decision criteria.

$\endgroup$
7
$\begingroup$

"Bayesian" and "frequentist" aren't "probabilistic philosophies". They're schools of statistical thought and practice concerned mainly with quantifying uncertainty and making decisions, although they're often associated with particular interpretations of probability. Probably the most common perception, although it is incomplete, is that of probability as subjective quantification of belief versus probabilities as long-run frequencies. But even these aren't really mutually exclusive. And you may not be aware of this but there are avowed Bayesians who don't agree on particular philosophical issues about probability.

Bayesian statistics and frequentist statistics aren't orthogonal either. It seems like "frequentist" has come to mean "not Bayesian" but that's incorrect. For example, it's perfectly reasonable to ask questions about the properties of Bayesian estimators and confidence intervals under repeated sampling. It's a false dichotomy perpetuated at least in part by a lack of a common definition of the terms Bayesian and frequentist (we statisticians have no one to blame but ourselves for that).

For an amusing, pointed and thoughtful discussion I would suggest Gelman's "Objections to Bayesian Statistics", the comments, and the rejoinder, available here:

http://ba.stat.cmu.edu/vol03is03.php

There is even some discussion about confidence intervals in physics IIRC. For more in-depth discussions you could walk back through the references therein. If you want to understand the principles behind Bayesian inference, I would suggest Bernando & Smith's book but there are many, many other good references.

$\endgroup$
6
$\begingroup$

Take a look at this paper by Cosma Shalizi and Andrew Gelman about philosophy and Bayesianism. Gelman is a proeminent bayesian and Shalizi a frequentist!

Take a look also at this short criticism by Shalizi, where he points the necessity of model checking and mock the dutch book argument used by some Bayesians.

And last, but not least, I think that, since you are a physicist, you may like this text, where the author points to “computational learning theory” (which I frankly know nothing at all), which could be an alternative to Bayesianism, as far as I can undestand it (not much).

ps.: If you follow the links, specially tha last one and have an opinion about the text (and the discussions that followed the text at the blog of the author)

ps.2: My own take on this: Forget about the issue of objective vs subjective probability, the likelihood principle and the argument about the necessity of being coherent. Bayesian methods are good when they allow you to model your problem well (for instance, using a prior to induce unimodal posterior when there is a bimodal likelihood etc.) and the same is true for frequentist methods. Also, forget about the stuff about the problems with p-value. I mean, p-value sucks, but in the end they are a measure of uncertainty, in the spirit of how Fisher thougth of it.

$\endgroup$
5
$\begingroup$

There are non-Bayesian systems or philosophies of probability -- Baconian & Pascalian, e.g. If you are into epistemology & philosophy of science you might enjoy the debates--otherwise, you'll shake your head & conclude that in fact the Bayesian interpretation is all there is.

For good discussions,

  • Cohen, L.J. An introduction to the philosophy of induction and probability, (Clarendon Press ; Oxford University Press, Oxford New York, 1989)
  • Schum, D.A. The evidential foundations of probabilistic reasoning, (Wiley, New York, 1994).
$\endgroup$
5
$\begingroup$

For me, the important thing about Bayesianism is that it regards probability as having the same meaning we apply intuitively in everyday life, namely the degree of plausibility of the truth of a proposition. Very few of us really use probability to mean strictly a long run frequency in everyday use, if only because we are often interested in particular events that have no long run frequency, for example what is the probability that fossil fuel emissions are causing significant climate change? For this reason, Bayesian statistics are much less prone to misinterpretation than frequentist statistics.

Bayesianism also has marginalisation, priors, maxent, transformation groups etc. that all have their uses, but for me the key benefit is that the definition of probability is more appropriate for the kinds of problems I want to address.

That doesn't make Bayesian statistcs better than frequentist statistics. It seems to me that frequentist statistics are well suited to problems in quality control (where you do have repeated sampling from populations) or where you have designed experiments, rather than analysis of pre-collected data (although that lies rather beyond my expertise, so it is just intuition).

As an engineer, it is a matter of "horses for courses" and I have both sets of tools in my toolbox and I use both on a regular basis.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.