40
$\begingroup$

As the question states - Is it possible to prove the null hypothesis? From my (limited) understanding of hypothesis, the answer is no but I can't come up with a rigorous explanation for it. Does the question have a definitive answer?

Improve this question
$\endgroup$
4

8 Answers 8

Reset to default
21
$\begingroup$

If you are talking about the real world & not formal logic, the answer is of course. "Proof" of anything by empirical means depends on the strength of the inference one can make, which in turn is determined by validity of the testing process as evaluated in light of everything one knows about how the world works (i.e., theory). Whenever one accepts that certain empirical results justify rejecting the "null" hypothesis, one is necessarily making judgments of this sort (validity of design; world works in certain way), so having to make the analogous assumptions necessary to justify inferring "proof of the null" is not problematic at all.

So what are the analogous assumptions? Here is an example of "proving the null" that is commonplace in health science & in social science. (1) Define "null" or "no effect" in some way that is practically meaningful. Let's say that I believe that I should conduct myself as if there is no meaningful difference between 2 treatments, t1 & t2, for a disease unless one gives a 3% better chance of recovery than the other. (2) Figure out a valid design for testing whether there is any effect-- in this case, whether there is a difference in recovery likelihood between t1 & t2. (3) Do a power analysis to determine whether what sample size is necessary to generate a sufficiently high likelihood-- one that I am confident relying on given what's at stake -- that I would see the meaningful effect, 3% in my example, assuming it exists. Usually people say power is sufficient if the likelihood of observing a specified effect at a specified alpha is at least 0.80, but the right level of confidence is really a matter of how averse you are to error -- same as it is when you select p-value threshold for "rejecting the null."(4) Perform the empirical test & observe the effect. If it is below the specified "meaningful difference" value -- 3% in my example -- you've "proven" that there is "no effect."

For a good treatment of this matter, see Streiner, D.L. Unicorns Do Exist: A Tutorial on “Proving” the Null Hypothesis. Canadian Journal of Psychiatry 48, 756-761 (2003).

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ +1. This is a nice example of the importance of being clear about one's standard of "proof." In many applications the one you invoke here--the "act as if" standard, if I may call it that--is so weak that nobody would accept it as "proof." I do not deny its utility, though, and advocate this kind of approach to support rational decision making. (But maybe Bayesian methods are better... :-) $\endgroup$
    – whuber ♦
    Commented Jan 14, 2011 at 0:47
  • 1
    $\begingroup$ (+1) Nice answer. I added a link to an online version of Streiner's article; I hope you don't mind (feel free to remove). $\endgroup$
    – chl
    Commented Jan 14, 2011 at 9:18
  • 1
    $\begingroup$ couple more things: (1) Treating failure to reject the null as evidence in support of null is a shockingly common error & usual occasion for Streiner's point. This mistake essentially turns the strong aversion to type 1 error in "p < 0.05" norm into license to make type 2. S says, "wait--you need power..." (2) Whuber cites Hume's famous argument. H's pt is actually just as subversive of empirical proofs rejecting the null as of proofs of the null. H says induction can't support causal inference. Ok; but there's no alternative for empirical study! Go Pearl (& Bayes), not Hume, on causality! $\endgroup$
    – dmk38
    Commented Jan 14, 2011 at 23:49
  • 1
    $\begingroup$ this question on equivalence testing also has some good suggestions stats.stackexchange.com/questions/3038/… $\endgroup$
    – Jeromy Anglim
    Commented Feb 11, 2011 at 4:46
  • $\begingroup$ Is this equivalent to assuming "not the null" as the new null hypothesis and then rejecting this new null hypothesis? $\endgroup$
    – user37312
    Commented Feb 12, 2014 at 12:17
16
$\begingroup$

Answer from the mathematical side : it is possible if and only if "hypotheses are mutually singular".

If by "prove" you mean have a rule that can "accept" (should I say that:) ) $H_0$ with a probability to make a mistake that is zero, then you are searching what could be called "ideal test" and this exists:

If you are testing wether a random variable $X$ is drawn from $P_0$ or from $P_1$ (i.e testing $H_0: X\leadsto P_0$ versus $H_1: X\leadsto P_1$) then there exists an ideal test if and only if $P_1\bot P_0$ ($P_1$ and $P_0$ are "mutually singular").

If you don't know what "mutually singular" means I can give you an example: $\mathcal{U}[0,1]$ and $\mathcal{U}[3,4]$ (uniforms on $[0,1]$ and $[3,4]$) are mutually singular. This means if you want to test

$H_0: X\leadsto \mathcal{U}[0,1]$ versus $H_1: X\leadsto \mathcal{U}[3,4]$

then there exist an ideal test (guess what it is :) ) : a test that is never wrong !

If $P_1$ and $P_0$ are not mutually singular, then this does not exist (this results from the "only if part")!

In non mathematical terms this means that you can prove the null if and only if the proof is already in your assumptions (i.e. if and only if you have chosen the hypothesis $H_0$ and $H_1$ that are so different that a single observation from $H_0$ cannot be identifyed as one from $H_1$ and vise versa).

Improve this answer
$\endgroup$
1
  • 4
    $\begingroup$ +1 Nice answer. A simple rendering of the math is that the null and its alternatives are assumed to yield disjoint sets of outcomes; e.g., either there is a zebra in this room or there isn't. Of course "prove" here implicitly includes "conditional on the model," which itself is never established with the same rigor as, say, a mathematical theorem; it implicitly includes "conditional on the accuracy of the observations;" and it implicitly includes that the hypotheses can be unambiguously interpreted. (For criticism of the latter, see George Lakoff's Women, Fire, and Dangerous Things.) $\endgroup$
    – whuber ♦
    Commented Jan 13, 2011 at 21:29
12
$\begingroup$

Yes there is a definitive answer. That answer is: No, there isn't a way to prove a null hypothesis. The best you can do, as far as I know, is throw confidence intervals around your estimate and demonstrate that the effect is so small that it might as well be essentially non-existent.

Improve this answer
$\endgroup$
2
  • 5
    $\begingroup$ More generally the problem in statistics is not that you are unable to prove the null hypothesis, it is that you are not able to make any point estimates with certainty. That is, just as you can't say "there is no effect of the variable" you are unable to say that "the effect size of the variable is 1.95". Statistics always have confidence intervals. $\endgroup$
    – russellpierce
    Commented Jan 20, 2011 at 3:41
  • 1
    $\begingroup$ Agreed that the answer is a big NO, and for a very strong reason: by construction of statistical hypothesis. The fact that the accepted answer is claiming otherwise is absolutely tragic. What hypothesis testing provides as an answer is: assuming my hypothesis is true, are the data that I sampled consistent with it? And by no means the other way around. It does not take much reasoning to understand that you cannot deduce from that whether the hypothesis is true or not. $\endgroup$
    – Christophe
    Commented Mar 27, 2019 at 7:56
7
$\begingroup$

For me, the decision theoretical framework presents the easiest way to understand the "null hypothesis". It basically says that there must be at least two alternatives: the Null hypothesis, and at least one alternative. Then the "decision problem" is to accept one of the alternatives, and reject the others (although we need to be precise about what we mean by "accepting" and "rejecting" the hypothesis). I see the question of "can we prove the null hypothesis?" as analogous to "can we always make the correct decision?". From a decision theory perspective the answer is clearly yes if

1)there is no uncertainty in the decision making process, for then it is a mathematical exercise to work out what the correct decision is.

2)we accept all the other premises/assumptions of the problem. The most critical one (I think) is that the hypothesis we are deciding between are exhaustive, and one (and only one) of them must be true, and the others must be false.

From a more philosophical standpoint, it is not possible to "prove" anything, in the sense that the "proof" depends entirely on the assumptions / axioms which lead to that "proof". I see proof as a kind of logical equivalence rather than a "fact" or "truth" in the sense that if the proof is wrong, the assumptions which led to it are also wrong.

Applying this to the "proving the null hypothesis" I can "prove" it to be true by simply assuming that it is true, or by assuming that it is true if certain conditions are meet (such as the value of a statistic).

Improve this answer
$\endgroup$
4
$\begingroup$

Technically, no, a null hypothesis cannot be proven. For any fixed, finite sample size, there will always be some small but nonzero effect size for which your statistical test has virtually no power. More practically, though, you can prove that you're within some small epsilon of the null hypothesis, such that deviations less than this epsilon are not practically significant.

Improve this answer
$\endgroup$
4
$\begingroup$

There is a case where a proof is possible. Suppose you have a school and your null hypothesis is that the numbers of boys and of girls is equal. As the sample size increases, the uncertainty in the ratio of boys to girls tends to reduce, eventually reaching certainty (which is what I assume you mean by proof) when the whole pupil population is sampled.

But if you do not have a finite population, or if you are sampling with replacement and cannot spot resampled individuals, then you cannot reduce the uncertainty to zero with a finite sample.

Improve this answer
$\endgroup$
3
$\begingroup$

Yes, it is possible to prove the null--in exactly the same sense that it is possible to prove any alternative to the null. In a Bayesian analysis, it is perfectly possible for the odds in favor of the null versus any of the proposed alternatives to it to become arbitrarily large. Moreover, it is false to assert, as some of the above answers assert, that one can only prove the null if the alternatives to it are disjoint (do not overlap with the null). In a Bayesian analysis every hypothesis has a prior probability distribution. This distribution spreads a unit mass of prior probability out over the proposed alternatives. The null hypothesis puts all of the prior probability on a single alternative. In principle, alternatives to the null may put all of the prior probability on some non-null alternative (on another "point"), but this is rare. In general, alternatives hedge, that is, they spread the same mass of prior probability out over other alternatives--either to the exclusion of the null alternative, or, more commonly, including the null alternative. The question then becomes which hypothesis puts the most prior probability where the experimental data actually fall. If the data fall tightly around where the null says they should fall, then it will be the odds-on favority (among the proposed hypotheses) EVEN THOUGH IT IS INCLUDED IN (NESTED IN, NOT MUTUALLY EXCLUSIVE WITH) THE ALTERNATIVES TO IT. The believe that it is not possible for a nested alternative to be more likely than the set in which it is nested reflects the failure to distinguish between probability and likelihood. While it is impossible for a component of a set to be less probable than the entire set, it is perfectly possible for the posterior likelihood of a component of a set of hypotheses to be greater than the posterior likelihood of the set as a whole. The posterior likelihood of an hypothesis is the product of the likelihood function and the prior probability distribution that the hypothesis posits. If an hypothesis puts all of the prior probability in the right place (e.g., on the null), then it will have a higher posterior likelihood than an hypothesis that puts some of the prior probability in the wrong place (not on the null).

Improve this answer
$\endgroup$
0
$\begingroup$

I would like to discuss here a point a lot of users are somewhat confused. What is the real meaning of the Null Hypothesis statement H0: p=0? Are we trying to determine if the parameter p is zero? Of course not, there is no way to achieve such a goal.

What we intend to establish is that, given the data set, the evaluated parameter value is (or not) indiscernible from zero. Remember that NHST is "unfair" towards the alternative hypotheses: the null is ascribed a 95% Confidence Level, and only 5% to the alternative. In consequence a “non-significant" result does not mean that H0 holds but simply that we did not found sufficient evidence that the alternative is likely.

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.