7
$\begingroup$

Based on Neyman-Pearson Hypothesis Testing Theory, will the failure to reject a null hypothesis imply that one should accept the null hypothesis?

(It would be absolutely absurd to have this claim if one adopts Fisher's null hypothesis testing theory, but would not it make sense if one chooses NPHT?)

$\endgroup$
6
$\begingroup$

If "the Neymann–Pearson approach" is understood narrowly as extending Fisher's by introducing an alternative hypothesis in addition to the null hypothesis, then there's no motive to change the terminology. The alternative can influence only the choice of test statistic (by consideration of the test's power); once that choice has been made the distribution of the test statistic is calculated under the null. "Fail to reject" reflects this asymmetry between the null & alternative hypotheses; you're provisionally assuming the null true till amassing enough evidence to the contrary. ("Retain" is an alternative way of putting it.) Pedagogues labour this nice semantic distinction in an attempt to avert the misconception that an "insignificant" result necessarily reflects a preponderance of evidence against the alternative. For example, consider a single observation from a Gaussian distribution with unit variance & unknown mean, $X\sim\mathcal{N}(\mu,1)$. With the point null & alternative $H_0: \mu=0$ vs $H_\mathrm{A}: \mu=1$, a test of size $0.05$ rejects the null only when $x > 1.64$, even though the alternative is better supported whenever $x > 0.5$.

If on the other hand you were to take the decision-theoretic framework of the NP approach seriously (see the excellent answer here), you just wouldn't bother to perform a test that was underpowered for your purposes; then talk of "accepting" the null hypothesis would seem a lot more sensible. Some notable expositors of testing theory from this viewpoint have apparently thought so. Lehmann & Romano (2005), Testing Statistical Hypotheses, unabashedly use "accept" & "reject" throughout. Casella & Berger (2002), Statistical Inference, use "accept" & "reject" too, even saying "We view a hypothesis testing problem as a problem in which one of two actions is going to be taken—the actions being the assertion of $H_0$ or $H_1$".

† An asymmetry exacerbated when the null's composite—either through specifying a range of parameter values, or because of a nuisance parameter that hasn't been removed through conditioning on an ancillary statistic— in which case it isn't rejected unless the test statistic is sufficiently extreme under any of its constituent simple nulls.

‡ While Cox & Hinkley (1974) Theoretical Statistics, manage four chapters on testing without using "accept" or "reject", except once in scare quotes.

$\endgroup$
  • 1
    $\begingroup$ As @gung put it in his excellent answer that you link to, "The central contention of the Neyman-Pearson framework is that at the end of your study, you have to make a decision and walk away". The decision here is to reject or not to reject the null. But if in this approach we are going to make a decision "and walk away", then isn't "not rejecting" basically the same as "accepting"? Quoting now your answer, "say[ing] that the data don't give you a strong indication of the direction of the effect" sounds rather Fisherian and has this "go and collect more data" flavour. $\endgroup$ – amoeba says Reinstate Monica Jan 5 '15 at 17:33
  • 1
    $\begingroup$ @amoeba: I suppose if you were to take the decision-theoretic framework of the NP approach seriously, you just wouldn't bother to perform a test that was underpowered for your purposes; then talk of "accepting" the null hypothesis would seem a lot more sensible. I have just been a little surprised to see Lehmann uses "accept", as do Casella & Berger (authors firmly in the NP camp); flicking through Cox & Hinkley, I didn't see either "accept" or "reject". $\endgroup$ – Scortchi - Reinstate Monica Jan 6 '15 at 11:36
  • 1
    $\begingroup$ Yes, absolutely! As I understand the NP approach, you should only run the test once you collected enough data to have the desired power. Then failing to reject means that the effect is too small to care about (with a certain pre-specified error rate of course), and it might be valid to say that we "accept" the null. Even though there might still be a subtle linguistic point in insisting to call it "failing to reject" rather than "accept". +1, but I would appreciate if you extend your answer with discussing this issue of power (as that is what I think the OP was largely about). $\endgroup$ – amoeba says Reinstate Monica Jan 6 '15 at 11:46
  • 1
    $\begingroup$ @amoeba: In my answer I was considering a kind of minimally NP approach - choosing a test statistic to have maximum power against a specified alternative. I will edit it - thanks for your help. $\endgroup$ – Scortchi - Reinstate Monica Jan 6 '15 at 11:55
  • $\begingroup$ To be honest, I never fully understood the connection between "NP approach" (as opposed to the Fisher approach) and "NP lemma" (that has to do with choosing a test statistic). Here on CV we only have a [neyman-pearson-lemma] tag, and the questions about NP approach (like this one) are often tagged with it. Does it even make sense? Isn't NP "approach" quite separate from NP lemma? Or am I missing something here? Apologies for this side-issue. $\endgroup$ – amoeba says Reinstate Monica Jan 6 '15 at 12:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.