Summary: There appears to be scientists that refuse to put prior probabilities on some statements, such as the existence of the Higgs Boson. This is an understandable position. These scientists, however, will not shun from claiming to have discovered something when they hit p = $\frac{1}{3,500,000}$ (five sigmas). Yet, if they claim a discovery, certainly they believe it is probably true. But unless they put a prior probability on their statement, they cannot obtain a posterior one, and they can hence not claim the statement is probably true. But they do. Where's the logic in this? :)

Why can scientists that refuse to fix a prior probability (or a lower bound on the prior) declare discoveries?

Take the discovery of the Higgs Boson in 2012 (Scientific American emphasis mine):

Chances are, you heard this month about the discovery of a tiny fundamental physics particle that may be the long-sought Higgs boson. The phrase five-sigma was tossed about by scientists to describe the strength of the discovery. So, what does five-sigma mean? In short, five-sigma corresponds to a p-value, or probability, of $3\times10^7$, or about 1 in 3.5 million. This is not the probability that the Higgs boson does or doesn't exist; rather, it is the probability that if the particle does not exist, the data that CERN scientists collected in Geneva, Switzerland, would be at least as extreme as what they observed. "The reason that it's so annoying is that people want to hear declarative statements, like 'The probability that there's a Higgs is 99.9 percent,' but the real statement has an 'if' in there. There's a conditional. There's no way to remove the conditional," says Kyle Cranmer, a physicist at New York University [...]

The statement in bold is a refusal to go Bayesian and assign a prior.

With a five sigma p-value, very few people would bet one million dollars that the Higgs Boson does not exist, yet many would bet all in that it does exist. This means that the unconditional posterior probability that the Higgs Boson exists is much greater than 50% to these (I believe rational) people:

$$\Pr(Higgs|Data) \gg 0.50$$

In 2013, the European Council for Nuclear Research (CERN) officially confirmed the existence of the Higgs Boson. If the word "confirmed" means anything at all, then certainly, at the very least, the CERN feels that

$$\Pr(Higgs|Data) > 0.90$$

But then using Bayes' theorem, given five sigma (i.e., p = 0.00000029) we can put a non-zero lower bound on the prior probability, Pr(Higgs), that the CERN used to confirm the existence of the Higgs Boson:

$$\scriptsize \Pr(Higgs \mid Data) = \frac{Pr(Data \mid Higgs) \,\times \Pr(Higgs)}{Pr(Data \mid Higgs)\times \Pr(Higgs) + \Pr(Data \mid Not\:Higgs)\times (1-\Pr(Higgs))} $$

$$ 0.90 < \frac{1 \times \Pr(Higgs)}{\Pr(Higgs) + .00000029\times (1 - \Pr(Higgs))} $$

$$ Pr(Higgs) > 0.00000289999 $$

Consciously or not they believed reasonable and used a prior at least this high. Using this same prior, if one day the CERN obtained 10-sigma confidence, they could (contra Kyle Cranmer) say something like "the probability that a Higgs-like particle exists is above 99.9%".

What part of this reasoning might Kyle Cranmer not like? What is the point of getting data to 150-sigma if you cannot at some point convert the conditional probability into an unconditional one? How can presumably serious scientists like Kyle Cranmer get away with apparently believing both of the following seemingly mutually exclusive statements:

  1. There's no way to estimate the probability that the Higgs exists.
  2. The Higgs probably exists.
  • 1
    $\begingroup$ I lost you at the summary phrase "probably true." These scientists have seen a signal rising above the noise, they know it's unlikely such a signal could be produced by pure noise, and this signal is nicely explained by their current theories. That's a form of discovery. "Probably exists," if it has any meaning at all, must be understood within a framework of a particular theory (for otherwise "Higgs Boson" is a meaningless phrase) and a set of experiments--but it doesn't seem to be a statistical conclusion, but more of a psychological one. $\endgroup$
    – whuber
    Jul 7, 2017 at 20:33
  • $\begingroup$ @whuber It's a discovery only if you admit a prior that's large enough. For every signal, regardless of its strength, there exists priors sufficiently small (at the limit, a dogmatic prior of zero) so that discovery cannot be claimed. In this case, say 10^-43. Are 10^-43 or 10^-2500 legitimate priors? If they are (and those that won't bound the prior fear they might be), we definitely cannot claim discovery. But we do. Why? $\endgroup$
    – Pertinax
    Jul 7, 2017 at 20:45
  • $\begingroup$ I don't see why you need a prior distribution. If you insist you do and you further insist that priors of the type you describe are plausible, then you are in a quandary, because you will be unable to do science or make decisions. $\endgroup$
    – whuber
    Jul 7, 2017 at 21:21


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.