I had a discussion with a statistician back in 2009 where he stated that the exact value of a p-value is irrelevant: the only thing that is important is whether it is significant or not. I.e. one result cannot be more significant than another; your samples for example, either come from the same population or don't.
I have some qualms with this, but I can perhaps understand the ideology:
The 5% threshold is arbitrary, i.e. that p = 0.051 is not significant and that p = 0.049 is, shouldn't really change the conclusion of your observation or experiment, despite one result being significant and the other not significant.
The reason I bring this up now is that I'm studying for an MSc in Bioinformatics, and after talking to people in the field, there seems to be a determined drive to get an exact p-value for every set of statistics they do. For instance, if they 'achieve' a p-value of p < 1.9×10-12, they want to demonstrate HOW significant their result is, and that this result is SUPER informative. This issue exemplified with questions such as: Why can't I get a p-value smaller than 2.2e-16?, whereby they want to record a value that indicates that by chance alone this would be MUCH less than 1 in a trillion. But I see little difference in demonstrating that this result would occur less than 1 in a trillion as opposed to 1 in a billion.
I can appreciate then that p < 0.01 shows that there is less than 1% chance that this would occur, whereas p < 0.001 indicates that a result like this is even more unlikely than the aforementioned p-value, but should your conclusions drawn be completely different? After all they are both significant p-values. The only way I can conceive of wanting to record the exact p-value is during a Bonferroni correction whereby the threshold changes due to the number of comparisons made, thus decreasing the type I error. But even still, why would you want to show a p-value that is 12 orders of magnitude smaller than your threshold significance?
And isn't applying the Bonferroni correction in itself slightly arbitrary too? In the sense that initially the correction is seen as very conservative, and therefore there are other corrections that one can choose to access the significance level that the observer could use for their multiple comparisons. But because of this, isn't the point at which something becomes significant essentially variable depending upon what statistics the researcher wants to use. Should statistics be so open to interpretation?
In conclusion, shouldn't statistics be less subjective (although I guess the need for it to be subjective is as a consequence of a multivariate system), but ultimately I want some clarification: can something be more significant than something else? And will p < 0.001 suffice in respect to trying to record the exact p-value?