Origin of “5$\sigma$” threshold for accepting evidence in particle physics?

Question

News reports say that CERN will announce tomorrow that the Higgs boson has been experimentally detected with 5$\sigma$ evidence. According to that article:

5$\sigma$ equates to a 99.99994% chance that the data the CMS and ATLAS detectors are seeing aren’t just random noise — and a 0.00006% chance that they’ve been hoodwinked; 5$\sigma$ is the necessary certainty for something to be officially labeled a scientific “discovery.”

This isn't super rigorous, but it seems to say that physicists use standard "hypothesis testing" statistical methodology, setting $\alpha$ to $0.0000006$, which corresponds to $z=5$ (two-tailed)? Or is there some other meaning?

In much of science, of course, setting alpha to 0.05 is done routinely. This would be equivalent to "two-$\sigma$" evidence, although I've never heard of it being called that. Are there other fields (besides particle physics) where a much stricter definition of alpha is standard? Anyone know a reference for how the five-$\sigma$ rule got accepted by particle physics?

Update: I'm asking this question for a simple reason. My book Intuitive Biostatistics (like most stats books) has a section that explains how arbitrary the usual "P<0.05" rule is. I'd like to add this example of a scientific field where a much (much!) smaller value of $\alpha$ is considered necessary. But if the example is actually more complicated, with use of Bayesian methods (as some comments below suggest), then it wouldn't be quite apt or would require a lot more explanation.

Answer 1

In most applications of statistics there is that old chestnut about 'all models are wrong, some are useful'. This being the case, we would only expected a model to perform at a given level since we are describing some incredibly complicated process using some simple model.

Physics is very different, so intuition developed from statistical models isn't so appropriate. In Physics, in particular particle physics which deals directly with fundamental physical laws, the model really is supposed to be an exact description of reality. Any departure from what the model predicts must be completely explained by experimental noise, not a limitation of the model. This means that if the model is good and correct and the experimental apparatus understood the statistical significance should be very high, hence the high bar that is set.

The other reason is historical, the particle physics community has been burned in the past by 'discoveries' at lower significance levels being later retracted, hence they are generally more cautious now.

Answer 2

For a reason entirely different from that of physics, there are other fields with much more strict alphas when they engage in hypothesis testing. Genetic Epidemiology is among them, especially when they use "GWAS" (Genome-Wide Association Study) to look at various genetic markers for disease.

Because a GWAS study is a massive exercise in multiple hypothesis testing, the state-of-the-art analysis techniques are all built around much more strict alphas than 0.05. Other such "candidate screening" study techniques that follow in the wake of the genomics studies will likely do the same.

Answer 3

The level is so high to avoid premature announcements of news that later turns out to be spurious. For more discussion on this, see

http://physics.stackexchange.com/questions/8752/standard-deviation-in-particle-physics?rq=1

http://physics.stackexchange.com/questions/31126/how-many-sigma-did-the-discovery-of-the-w-boson-have