Why can scientists that refuse to bound the prior probability declare discoveries? - Cross Validated most recent 30 from stats.stackexchange.com 2019-06-17T15:00:10Z https://stats.stackexchange.com/feeds/question/289415 http://www.creativecommons.org/licenses/by-sa/3.0/rdf https://stats.stackexchange.com/q/289415 1 Why can scientists that refuse to bound the prior probability declare discoveries? Pertinax https://stats.stackexchange.com/users/46128 2017-07-07T20:03:08Z 2017-07-07T20:26:16Z <p><strong>Summary</strong>: <em>There appears to be scientists that refuse to put prior probabilities on some statements, such as the existence of the Higgs Boson. This is an understandable position. These scientists, however, will not shun from claiming to have discovered something when they hit p = $\frac{1}{3,500,000}$ (five sigmas). Yet, if they claim a discovery, certainly they believe it is probably true. But unless they put a prior probability on their statement, they cannot obtain a posterior one, and they can hence not claim the statement is probably true. But they do. Where's the logic in this? :)</em> </p> <p><strong>Why can scientists that refuse to fix a prior probability (or a lower bound on the prior) declare discoveries?</strong></p> <p>Take the discovery of the Higgs Boson in 2012 (<a href="https://blogs.scientificamerican.com/observations/five-sigmawhats-that/" rel="nofollow noreferrer">Scientific American</a> emphasis mine):</p> <blockquote> <p>Chances are, you heard this month about the discovery of a tiny fundamental physics particle that may be the long-sought Higgs boson. The phrase five-sigma was tossed about by scientists to describe the strength of the discovery. So, what does five-sigma mean? In short, five-sigma corresponds to a p-value, or probability, of $3\times10^7$, or about 1 in 3.5 million. This is not the probability that the Higgs boson does or doesn't exist; rather, it is the probability that if the particle does not exist, the data that CERN scientists collected in Geneva, Switzerland, would be at least as extreme as what they observed. "The reason that it's so annoying is that people want to hear declarative statements, like 'The probability that there's a Higgs is 99.9 percent,' but the real statement has an 'if' in there. <strong>There's a conditional. There's no way to remove the conditional</strong>," says Kyle Cranmer, a physicist at New York University [...]</p> </blockquote> <p>The statement in bold is a refusal to go Bayesian and assign a prior.</p> <p>With a five sigma p-value, very few people would bet one million dollars that the Higgs Boson does not exist, yet many would bet all in that it does exist. This means that the unconditional posterior probability that the Higgs Boson exists is much greater than 50% to these (I believe rational) people:</p> <p>$$\Pr(Higgs|Data) \gg 0.50$$</p> <p>In 2013, the European Council for Nuclear Research (CERN) officially confirmed the existence of the Higgs Boson. If the word "confirmed" means anything at all, then certainly, at the very least, the CERN feels that </p> <p>$$\Pr(Higgs|Data) &gt; 0.90$$</p> <p>But then using Bayes' theorem, given five sigma (i.e., p = 0.00000029) we can put a non-zero lower bound on the prior probability, Pr(Higgs), that the CERN used to confirm the existence of the Higgs Boson:</p> <p>$$\scriptsize \Pr(Higgs \mid Data) = \frac{Pr(Data \mid Higgs) \,\times \Pr(Higgs)}{Pr(Data \mid Higgs)\times \Pr(Higgs) + \Pr(Data \mid Not\:Higgs)\times (1-\Pr(Higgs))}$$</p> <p>$$0.90 &lt; \frac{1 \times \Pr(Higgs)}{\Pr(Higgs) + .00000029\times (1 - \Pr(Higgs))}$$</p> <p>$$Pr(Higgs) &gt; 0.00000289999$$</p> <p>Consciously or not they believed reasonable and used a prior at least this high. Using this same prior, if one day the CERN obtained 10-sigma confidence, they could (contra Kyle Cranmer) say something like "the probability that a Higgs-like particle exists is above 99.9%".</p> <p>What part of this reasoning might Kyle Cranmer not like? What is the point of getting data to 150-sigma if you cannot at some point convert the conditional probability into an unconditional one? How can presumably serious scientists like Kyle Cranmer get away with apparently believing both of the following seemingly mutually exclusive statements:</p> <ol> <li>There's no way to estimate the probability that the Higgs exists. </li> <li>The Higgs probably exists.</li> </ol>