What to claim when we don't reject the null hypothesis? In testing a null vs an alternative, if we don't reject the null (e.g. the p-value is very big, bigger than the significance level), what is our conclusion?
Can we say that we accept the null, can we? I remember I heard that not rejecting the null doesn't mean accepting the null, but I might be wrong.
Or should we simply say that we don't reject the null?
 A: I would include three things:


*

*The phrase "Insufficient evidence to reject". Shows that with more evidence, e.g. more data, or repeating the experiment with a different random selection of data, you might have rejected.

*The significance level. At a higher significance level you might have rejected the null hypothesis.

*What your conclusion is. "The data is consistent with" is a good phrase here.
As an example, putting these altogether for an experiment to determine the effect of cognitive behavioural therapy (CBT) on addressing insomnia
"There was insufficient evidence to reject the null hypothesis that CBT increased the amount of sleep at the 5% significance level. The data is consistent with CBT having no effect on insomnia."
A: "We fail to reject the null" is the correct answer. Rather than say, for example, "there is no difference" we should write "no difference was detected".
Clearly, with not enough replicates or large measurement error you are likely to not be able to detect even large effects. So maybe the difference is there, but you failed to see it?
As Maarten Buis writes in the comment: "Absence of evidence is not evidence of absence". (Personally, I am careful with the word "evidence".)
