8
$\begingroup$

This is not a math question, this is a methodology and philosophy question.

Sometimes I see language describing things as "very statistically significant" because the p-value is extremely small.

But in the way that a p-value of 0.06 should not be called "almost significant", I wonder if a p-value of 0.00006 should also not be called "very significant".

Improve this question
$\endgroup$
3
  • 7
    $\begingroup$ This sounds like a purely subjective matter to me. Although statistics teaches us to respect quantitative assertions over qualitative ones, anybody may exaggerate and use hyperbolic language. It's up to their audience to see that there's nothing in it. After all, a recently elected high government official in my country thinks our cities are a "huuuuge disaster," all evidence to the contrary. $\endgroup$
    – whuber ♦
    Jan 27, 2017 at 0:29
  • 1
    $\begingroup$ I sometimes use the term if the p-value is 0 by rounding errors. But I prefer "highly". $\endgroup$
    – ABCD
    Jan 27, 2017 at 1:06
  • 1
    $\begingroup$ Depends on the discipline. My former biostatistics professors used to say there's a threshold you define a priori (usually 0.05), and if it crosses that threshold, then it's "significant," otherwise, it is not ("otherwise it's a fishing expedition"). Professors in other disciplines sometimes would say "marginally not significant" if it was 0.06, but I felt at times they were "reaching" because they really wanted an association between a and b. And completely unrelated, @whuber, I believe what the official said was, "yuuuuuuge" $\endgroup$
    – Marquis de Carabas
    Jan 27, 2017 at 8:07

2 Answers 2

Reset to default
3
$\begingroup$

Significance testing has two different interpretations. The Neyman-Pearson interpretation is that a result is significant, or it is not significant, that is all that can be said. That is all you can say. The Fisher tradition is that a p-value is a representation of the strength of evidence against the null hypothesis - a p-value of 0.10 tells you something, a p-value of 0.01 also tells you something. Fisher wrote If

$P$ is between .1 and .9 there is certainly no reason to suspect the hypothesis tested. If it is below .02 it is strongly indicated that the hypothesis fails to account for the whole of the facts. We shall not often be astray if we draw a conventional line at .05 [...]

Improve this answer
$\endgroup$
0
$\begingroup$

if the p-value is around 0.05, we can say it is marginally significant, but p-value is very small (ex: 0.0006), then we can say it is statistically significant. I am not sure, you can use the term "very significant".

Improve this answer
$\endgroup$
2
  • 3
    $\begingroup$ Where did the number 0.05 come from? $\endgroup$
    – Matthew Drury
    Jan 27, 2017 at 1:41
  • $\begingroup$ @Matthew, If your confidence level is at 95%, you draw back conclusions based on the p-value greater or lesser than 0.05. It depends on what it is your confidence level. I would assume his question is considering 95% confidence level based on they way it is stated. $\endgroup$
    – NiroshaR
    Jan 30, 2017 at 15:23

Not the answer you're looking for? Browse other questions tagged or ask your own question.