Examples of studies using p < 0.001, p < 0.0001 or even lower p-values? I come from the social sciences, where p < 0.05 is pretty much the norm, with p < 0.1 and p < 0.01 also showing up, but I was wondering: what fields of study, if any, use lower p-values as a common standard?
 A: My opinion is that it does (and should) not depend on the field of study. For example, you may well work at a lower significance level than $p<0.001$ if, for example, you are trying to replicate a study with historical or well-established results (I can think of several studies on the Stroop effect, which had led to some controversies in the past few years). That amounts to consider a lower "threshold" within the classical Neyman-Pearson framework for testing hypothesis. However, statistical and practical (or substantive) significance is another matter.
Sidenote.
The "star system" seems to have dominated scientific inquiries as early as the 70's, but see The Earth Is Round (p < .05), by J. Cohen (American Psychologist, 1994, 49(12), 997-1003), despite the fact that what we often want to know is given the data I have observed, what is the probability that $H_0$ is true? Anyway, there's also a nice discussion on "Why P=0.05?", by Jerry Dallal.
A: It might be rare for anyone to use a pre-specified alpha level lower than, say, 0.01, but it is not nearly as rare that people claim an implied alpha of less than 0.01 in the mistaken belief that an observed P value of less than 0.01 is the same as a Neyman-Pearson alpha of less than 0.01.
Fisher's P values are not the same as, or interchangeable with, Neyman-Pearson error rates. $P = 0.0023$ does not mean $\alpha = 0.0023$ unless one has decided to use $0.0023$ as the critical level for significance when the experiment is designed. If you would have taken $P = 0.05$ as significant then $P = 0.0023$ means that there is an $0.05$ probability of a false positive claim.
Have a look at Hubbard et al. Confusion over Measures of Evidence (p's) versus Errors (α's) in Classical Statistical Testing. The American Statistician (2003) vol. 57 (3)
A: I am not very familiar with this literature but I believe some physicists use much lower thresholds in statistical tests. However, they talk about it a little differently so social scientists might not realise the connection.
For example, if a measure is three standard deviations from the theoretical prediction, it is described as a “three sigma” deviation. Basically, this means that the parameter of interest is statistically different from the predicted value in a z test with $α = .01$. Two sigma is roughly equivalent to $α = .05$ (in fact it would be 1.96 σ). If I am not mistaken, the standard error level in physics is 5 sigma, which would be $α = 5*10^-7$ or $p < 0.0000005$.
Also, in neuroscience or epidemiology, it seems increasingly common to routinely perform some correction for multiple comparisons. The error level for each individual test can therefore be lower than $p < .01$.
A: As noted by Gaël Laurans above statistical analyses that run into the multiple comparison problem tend to use more conservative thresholds. However, in essence they are using 0.05, but multiplied by the number of tests. It is obvious that this procedure (Bonferroni correction) can quickly lead to incredibly small p-values. That's why people in the past (in neuroscience) stopped at p<0.001. Nowadays other methods of multiple comparison corrections are used (see Markov random field theory).
