0
$\begingroup$

Let's take an example (Taken from "Naked Statistics by Charles Wheelan") :

  • Null hypothesis - Substance abuse treatment for prisoners does not reduce their rearrest rate after leaving prison
  • Alternative hypothesis - Substance abuse treatment for prisoners will make them less likely to be rearrested after they are released
  • The data - Prisoners were randomly assigned into two groups the "treatment" group received substance abuse treatment and the control group did not. At the end of the five years, both groups have similar rearrest rates
  • Inference - In this case, we cannot reject the null hypothesis

The author then proceeds to mention (In the footnotes) that

we have not proved the null hypothesis to be true. It may turn out to be extremely effective for another group of prisoners. Or perhaps many more of the prisoners in this treatment group would have been rearrested if they had not received the treatment. In any case, on the basis of the data collected, we have merely failed to reject our null hypothesis. There is a similar distinction between "failing to reject" a null hypothesis and accepting that null hypothesis. Just because one study could not disprove that substance abuse treatment has no effect does not mean that one must accept that substance abuse treatment is useless

I think I agree with the conclusion that is mentioned by the author in the footnotes though not for the reasons (in bold) that are given by the author.

I always thought that the reason for not accepting the null hypothesis is that we have not proven that precise statement of the null hypothesis to be true. In this study, if we had 4/100 (with treatment) and 5/100 (without treatment) rearrest rates from the two groups then we cannot reject the null hypothesis at a strict confidence level (say 99%) but neither have we proven that the treatment is completely useless. It could be any amount useful within the possibilities of that 99% confidence interval. How do make it more conclusive then? (say the study reduces rearrests by 20%. 5 brought down to 4 on average). We collect more data. 4/100 and 5/100 are easy to be obtained via chance but 400/10000 and 500/10000 are less likely and 4000/100000 and 5000/100000 even lesser likely But, not being to accept the null hypothesis because It may turn out to be extremely effective for another group of prisoners. Or perhaps many more of the prisoners in this treatment group would have been rearrested if they had not received the treatment is something that I cannot wrap my head around. This kind of reasoning cuts both ways as I see it. If we had 20/100 and 4/100 rearrests, then even while rejecting the null hypothesis we can interject that the second group could have anyway been less disposed to rearrest, to begin with. The study has to assume that it marginalizes over all the other factors and the effect differs only because of the cause hypothesized

  1. Is my understanding of why we don't accept the null hypothesis correct?
  2. How is the author's reasoning about why we don't accept the null hypothesis accurate?
Improve this question
$\endgroup$
12
  • 1
    $\begingroup$ I agree that the bold reasons are simply misstated and distracting from the actual problem. Your reasoning seems correct. To explain why accepting the null hypothesis after failing to reject is not possible, I prefer the nice saying "absence of evidence is not evidence of absence", which summarizes the philophical issue quite well. $\endgroup$
    – LuckyPal
    Commented Nov 12, 2021 at 10:48
  • 1
    $\begingroup$ In my opinion, the statement "absence of evidence is not evidence of absence" is wordplay without any scientific merit. How strong is an absence of evidence evidence of absence is what reliability engineering is, isn't it? The power of the test etc. If you think that your missing $100 is not in your pocket, you search for it and, you don't find it then you do have evidence that it actually isnt there. How rigorously you searched is the power of your search $\endgroup$
    – figs_and_nuts
    Commented Nov 12, 2021 at 11:14
  • 1
    $\begingroup$ One reason we do not accept the null hypothesis is that often we know that it is false a-priori, e.g. the unbiasness of a coin will not be precisely zero (or at least it is essentially infinitely unlikely to be exactly zero). I don't think statistics can be used to prove anything. $\endgroup$
    – Dikran Marsupial
    Commented Nov 12, 2021 at 14:44
  • 1
    $\begingroup$ Thank you @DikranMarsupial . The insight that we know the null to be false a-priori is something I liked. In all one-tailed tests though, is it not the case that we do not know the null to be false a-priori? Also, do you have any idea what the author has in mind while stating those reasons for why we don't accept the null hypothesis? $\endgroup$
    – figs_and_nuts
    Commented Nov 12, 2021 at 18:08
  • 2
    $\begingroup$ Some possible dups: stats.stackexchange.com/questions/85903/…, stats.stackexchange.com/questions/275677/…, stats.stackexchange.com/questions/163957/…, stats.stackexchange.com/questions/60670/… $\endgroup$
    – kjetil b halvorsen ♦
    Commented Nov 12, 2021 at 22:01

0

Reset to default