10
$\begingroup$

I'm trying to conduct a Student's t-test for a table of values while trying to follow the explanation and details found on this website. I understand that if the p-value is

  • <.01 then it's really significant
  • >.05 it's not significant
  • in between then we need more data

But on that page they seem to accept their null hypothesis no matter what the p-value is. So I'm really not understanding now when to accept or reject the null hypothesis.

  • When you do you accept or reject the null-hypothesis?
  • Is it true that you are never supposed to accept your null hypothesis, but rather reject or fail to reject the null?
$\endgroup$

marked as duplicate by Silverfish, whuber Dec 12 '15 at 17:17

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 7
    $\begingroup$ You are right: If the $p$-value is >.05, we often say that we fail to reject the null hypothesis or that we don't have evidence to suggest that the means are different. This does not mean that the null hypothesis is true. But they explain it on the website: "In science, when we accept a hypothesis, this does NOT mean we have decided that the hypothesis is correct or that it is probably correct." I doubt if "accept" is the best term in this case as it can lead to confusion. $\endgroup$ – COOLSerdash Jun 2 '13 at 16:57
  • 2
    $\begingroup$ This article here also advocates that the term "accept" should not be used by scientists. $\endgroup$ – COOLSerdash Jun 2 '13 at 17:19
  • 2
    $\begingroup$ The explanation on that website is very poorly worded. $\endgroup$ – Peter Flom Jun 2 '13 at 21:08
  • 2
    $\begingroup$ "fail to reject the null hypothesis" (or something similar) is the way I generally put it on the rare occasions when I formally test a hypothesis and don't reject the null. I almost never think the null has a chance to be actually true so it's more a lack of evidence against the null than in any sense an acceptance that the null is the case. $\endgroup$ – Glen_b Jun 3 '13 at 1:53
  • 6
    $\begingroup$ All this begs the questions of why we need formal hypotheses vs. estimating a quantity of interest and reporting confidence intervals. There is no bifurcation implied by estimation. $\endgroup$ – Frank Harrell Jul 15 '13 at 12:09
15
$\begingroup$

I would suggest that it is much better to say that we "fail to reject the null hypothesis", as there are at least two reasons we might not achieve a significant result: Firstly it may be because H0 is actually true, but it might also be the case that H0 is false, but we have not collected enough data to provide sufficient evidence against it. Consider the case where we are trying to determine whether a coin is biased (H0 being that the coin is fair). If we only observe 4 coin flips, the p-value can never be less than 0.05, even if the coin is so biased it has a head on both sides, so we will always "fail to reject the null hypothesis". Clearly in that case we wouldn't want to accept the null hypothesis as it isn't true. Ideally we should perform a power analysis to find out if we can reasonably expect to be able to reject the null hypothesis when it is false, however this isn't usually nearly as straightforward as performing the test itself, which is why it is usually neglected.

Update: The null hypothesis is quite often known to be false before observing the data. For instance a coin (being asymmetric) is almost certainly biased; the magnitude of this bias us undoubtedly negligible, but not precisely zero, which is what the H0 for the usual test of the bias of a coin asserts. If we observe a sufficiently large number of flips, we will eventually be able to detect this miniscule deviation from exact unbiasedness. It would be odd then to accept the "null hypothesis" in this case as we know before performing the test that it is certainly false. The test is certainly still useful though as we are generally interested in whether the coin is practically biased.

$\endgroup$
  • 1
    $\begingroup$ And what if the Power (from simulations) for that test is really high. Could I say "accept the Null"? $\endgroup$ – An old man in the sea. Feb 19 '15 at 13:30
  • $\begingroup$ That would be better, but there are still a lot that is left implicit, for instance H1 may not be the only alternative to H0 and the relative prior probabilities of H0 and H1 (which only really enter into the test indirectly via $\alpha$ and $\beta$) may be strongly against H0. The real problem is that we really want to know the probability that H0 is true, which a frequentist test can't give us, so it is a good idea to avoid terminology that could be interpreted that way, I like "fail to reject the null hypothesis" as it implies that its meaning is subtle (which it is!). $\endgroup$ – Dikran Marsupial Feb 19 '15 at 14:13
  • 3
    $\begingroup$ There is also the point that sometimes you know a-priori that the null hypothesis is certainly false (e.g. that a coin is exactly unbiased), and it would be odd to accept something that you know isn't true even before performing the test. $\endgroup$ – Dikran Marsupial Feb 19 '15 at 14:16

Not the answer you're looking for? Browse other questions tagged or ask your own question.