News reports say that CERN will announce tomorrow that the Higgs boson has been experimentally detected with 5$\sigma$ evidence. According to that article:
5$\sigma$ equates to a 99.99994% chance that the data the CMS and ATLAS detectors are seeing aren’t just random noise — and a 0.00006% chance that they’ve been hoodwinked; 5$\sigma$ is the necessary certainty for something to be officially labeled a scientific “discovery.”
This isn't super rigorous, but it seems to say that physicists use standard "hypothesis testing" statistical methodology, setting $\alpha$ to $0.0000006$, which corresponds to $z=5$ (two-tailed)? Or is there some other meaning?
In much of science, of course, setting alpha to 0.05 is done routinely. This would be equivalent to "two-$\sigma$" evidence, although I've never heard of it being called that. Are there other fields (besides particle physics) where a much stricter definition of alpha is standard? Anyone know a reference for how the five-$\sigma$ rule got accepted by particle physics?
Update: I'm asking this question for a simple reason. My book Intuitive Biostatistics (like most stats books) has a section that explains how arbitrary the usual "P<0.05" rule is. I'd like to add this example of a scientific field where a much (much!) smaller value of $\alpha$ is considered necessary. But if the example is actually more complicated, with use of Bayesian methods (as some comments below suggest), then it wouldn't be quite apt or would require a lot more explanation.