In scientific papers there are asterisks representing the significance. Do these asterisks represent the significance level of the performed test or do they represent the obtained p-values? For example if you perform a t-test with a significance level of 0.05 and you get a p-value of 0.003. Then you have to reject the null-hypothesis. But can you put one asterisk (p<0.05) or two asterisks (p<0.01) above the graph if you make one?
Do asterisks representing significance in figures refer to the p-value or to the significance level?
$\begingroup$
$\endgroup$
4
-
1$\begingroup$ I don't understand the question: what do you mean by "make one"? $\endgroup$– whuber ♦Apr 1, 2016 at 17:20
-
$\begingroup$ @whuber "if you make one" refers to the "the graph", i.e. "when one is making a figure, what are the rules for deciding on the asterisk". $\endgroup$– amoebaApr 1, 2016 at 21:25
-
$\begingroup$ @amoeba That interpretation occurred to me, but I gave it a low probability of being correct because asterisks are usually applied to tables, not graphs. I haven't any definite idea what kind of "graph" is being referred to here, either. It seems one has to make a lot of assumptions in order to follow this question, which is why I have specifically requested clarification. $\endgroup$– whuber ♦Apr 1, 2016 at 21:34
-
$\begingroup$ @whuber Oh, it is very very common in some fields to use asterisks in figures. Look at this: google.com/search?q=figure+asterisk+significance&tbm=isch. $\endgroup$– amoebaApr 1, 2016 at 21:35
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
5
There's not a single convention for asterisks. Sometimes they are for 10, 5 and 1% significance, or 5, 1 and 0.1% significance. Other times they could be in standard deviations and so on. You always have to read the table captions to see what they represent.
For instance, a table caption may say that the significance levels are given by stars: * - 10%, ** - 5% and *** - 1%. In this case a coefficient with ** would mean that the p-value was under 0.05. It's like Michelin rating - more stars, better. At least, that's what I see in papers.
-
$\begingroup$ (+1) But do they indicate achieving a prespecified significance level - so that if you were conducting all tests at a significance level of 5% you'd give one star only to all p-values under 5%, say, and no stars to all p-values over 5%? Or does one star indicate an observed p-value between 5% & 1%, say, & two stars indicate ones between 1% & 0.1%, &c.? I think that was part of the question. (And I don't recall having seen or heard of the former convention.) $\endgroup$ Apr 1, 2016 at 21:16
-
$\begingroup$ Conventions for how many asterisks corresponds to what number may differ, but asterisks always refer to the actual p-values, I agree with @Scortchi. $\endgroup$– amoebaApr 1, 2016 at 21:28
-
$\begingroup$ Giving one star to $p=0.1$ is weird. What fields are using this convention? $\endgroup$– amoebaApr 1, 2016 at 21:33
-
$\begingroup$ @amoeba, I do it in economic forecasting :) when you have 50 observations, it's a gift to get $\alpha=0.05$. The point is that it's not a convention. I put the mapping in the table caption, so the reader knows what's it about. My table captions tend to be long, so you don't have to read the text to browse the tables and plots. $\endgroup$– AksakalApr 1, 2016 at 21:37
-
$\begingroup$ @amoeba Apparently it is (was?) a thing in political science. If you have access to it, see Goldstein, J. S. (2010). On Asterisk Inflation. dx.doi.org/10.1017/S104909650999059X p.59: "One might consider the standard of one asterisk for .05 set in stone. However, in the past decade the use of asterisks has changed in political science. Today, frequently, one asterisk means p < .10. The magic level of p < .05 that used to receive an asterisk now receives two, and .01, which used to get two asterisks, now gets three. I call it asterisk inflation." $\endgroup$– J-J-JSep 5 at 9:00
$\begingroup$
$\endgroup$
1
I tend to find that the following notation is common in psychology papers, with the first and last rows being more rare than the middle two which are almost standard.
p <.0001 ***
p <.01 **
p <.05 *
p <.1 †
Where the latter (dagger/obelisk) is usually referred to as a non-significant "trend."
That said, even the centre two rows are only almost standard, not standard, so you should make it plain in the paper. In an oral conference presentation with slides. I would expect my audience to understand the meaning of the middle two rows however.
$\begingroup$
$\endgroup$
Trying to answer the question that was actually asked: it varies, but I would lean towards the achieved p-values rather than the pre-planned significance levels. (I personally would definitely use achieved p-values, but I think that's also more common in the literature). In written text there has been strong movement towards requiring explicit p-values rather than significance statements.
Two specific notes:
- Software that gives you stars will do it based on the p-value, because it usually doesn't know what significance levels you had in mind.
- In a publication, reporting (and basing stars on) the achieved p-values lets other people, who might be using different pre-specified significance levels or none, apply their own criteria
