In scientific papers there are asterisks representing the significance. Do these asterisks represent the significance level of the performed test or do they represent the obtained p-values? For example if you perform a t-test with a significance level of 0.05 and you get a p-value of 0.003. Then you have to reject the null-hypothesis. But can you put one asterisk (p<0.05) or two asterisks (p<0.01) above the graph if you make one?
There's not a single convention for asterisks. Sometimes they are for 10, 5 and 1% significance, or 5, 1 and 0.1% significance. Other times they could be in standard deviations and so on. You always have to read the table captions to see what they represent.
For instance, a table caption may say that the significance levels are given by stars: * - 10%, ** - 5% and *** - 1%. In this case a coefficient with ** would mean that the p-value was under 0.05. It's like Michelin rating - more stars, better. At least, that's what I see in papers.
I tend to find that the following notation is common in psychology papers, with the first and last rows being more rare than the middle two which are almost standard.
p <.0001 ***
p <.01 **
p <.05 *
p <.1 †
Where the latter (dagger/obelisk) is usually referred to as a non-significant "trend."
That said, even the centre two rows are only almost standard, not standard, so you should make it plain in the paper. In an oral conference presentation with slides. I would expect my audience to understand the meaning of the middle two rows however.
Trying to answer the question that was actually asked: it varies, but I would lean towards the achieved p-values rather than the pre-planned significance levels. (I personally would definitely use achieved p-values, but I think that's also more common in the literature). In written text there has been strong movement towards requiring explicit p-values rather than significance statements.
Two specific notes:
- Software that gives you stars will do it based on the p-value, because it usually doesn't know what significance levels you had in mind.
- In a publication, reporting (and basing stars on) the achieved p-values lets other people, who might be using different pre-specified significance levels or none, apply their own criteria