Timeline for If we fail to reject the null hypothesis in a large study, isn't it evidence for the null?
Current License: CC BY-SA 4.0
17 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 22, 2019 at 3:32 | history | edited | Mark White | CC BY-SA 4.0 |
added 9 characters in body
|
| Nov 6, 2017 at 16:02 | history | edited | Mark White | CC BY-SA 3.0 |
added 44 characters in body
|
| Apr 26, 2017 at 7:49 | history | edited | Tim | CC BY-SA 3.0 |
full reference
|
| Apr 26, 2017 at 2:56 | comment | added | Andrew Hill | A good example i have seen is when starting from a wrong assumption: If a person is american, he is NOT likely a member of congress. This person is in congress, therefore he is not likely american. | |
| Apr 25, 2017 at 15:16 | comment | added | bkoodaa | @MarkWhite How about "If the null is true, then a p-value above significance level 0.05 is likely"? | |
| Apr 25, 2017 at 14:27 | comment | added | Mark White | @StephanKolassa dang it, you're right; I should have remembered the distribution of p-values under the null from reading about the p-curve. | |
| Apr 25, 2017 at 11:49 | comment | added | Stephan Kolassa | "If the null is true, then a high p-value is likely." - this is not correct. If the null hypothesis is true, then $p\sim U[0,1]$, so high $p$ values are no more likely than low ones under the null hypothesis. All you can say is that a high $p$ value is more likely under the null than under other hypotheses - but the hypotheses either hold or don't, so the hypotheses are not the probability space in which we are operating. Unless we work in a Bayesian paradigm! And that is where your argument unfortunately breaks down. | |
| Apr 25, 2017 at 11:48 | comment | added | amoeba | I disagree with your conclusion "If we want evidence for the null, Bayesian inference is required"; the study that you are citing is from Wagenmakers who is a very vocal hard-core proponent of Bayesian statistics so obviously they argues that. But in fact one can easily have evidence "for the null" in the frequentist paradigm, e.g. by conducting TOST (two one-sided tests) for equivalence. (cc @AtteJuvonen). | |
| Apr 25, 2017 at 8:43 | comment | added | Tim | @AtteJuvonen yes, we have some evidence for rain, but we do not know how likely it is, so the only conclusion that you can make is that "it could have rained, or it could have been something else that made ground wet". So you have inconclusive evidence. Only on the grounds of Bayesian statistics you can make the opposite argument. | |
| Apr 25, 2017 at 8:17 | comment | added | bkoodaa | ...Unless the ground is wet literally 100% of the time. In that case we can't find evidence AGAINST the null hypothesis, either, so that special case isn't really relevant to this question. If we have a reasonable study where we can find evidence against the null hypothesis, we can also find evidence for the null hypothesis. | |
| Apr 25, 2017 at 8:11 | comment | added | bkoodaa | @Tim I never said that. Please stop creating strawman arguments. If we observe that the ground is wet, then we have evidence for "rain". Having this evidence means that the probability of rain is HIGHER than it would be without that evidence. | |
| Apr 25, 2017 at 7:26 | comment | added | Tim | @AtteJuvonen so if it rains, say 1% time of the year, and people by going in and out the swimming pole make everything around wet all the time, then you would still claim that wet ground makes you believe that it was recently raining..? | |
| Apr 25, 2017 at 6:45 | comment | added | bkoodaa | @Tim Even if one lives near the swimming pool, a wet ground makes it somewhat likelier that rain has happened. I am not claiming that P(A|B) = P(B|A) so I don't know why you would bring that up. | |
| Apr 25, 2017 at 6:38 | comment | added | Tim | @AtteJuvonen probability may be seen as an extension of logic with extra uncertainty added. When you say that wet ground makes rain more likely you make exactly the described mistake. What if someone lives near to swimming pole on the desert? You ignore the prior probability of rain! $P(A|B) \ne P(B|A)$. | |
| Apr 25, 2017 at 5:55 | comment | added | Mark White | That's fair. That Rouder et al. paper I linked to at the end of my answer does not have examples that have conclusions with certainty. | |
| Apr 25, 2017 at 5:48 | comment | added | bkoodaa | I don't like that all of your examples conclude "X is true". Having evidence for something is not the same thing as concluding something with 100% certainty. If I go outside and the ground is wet, that is evidence for "it rained". That evidence makes it much more likely that rain has occurred. | |
| Apr 25, 2017 at 5:29 | history | answered | Mark White | CC BY-SA 3.0 |