Traditionally, the null hypothesis is a point value. (It is typically $0$, but can in fact be any point value.) The alternative hypothesis is that the true value is any value other than the null value. Because a continuous variable (such as a mean difference) can take on a value which is indefinitely close to the null value, but still not quite equal and thus make the null hypothesis false, a traditional point null hypothesis cannot be proven.

Imagine your null hypothesis is $0$, and the mean difference you observe is $0.01$. Is it reasonable to assume the null hypothesis is true? You don't know yet; it would be helpful to know what our confidence interval looks like. Let's say that your 95% confidence interval is $(-4.99,\ 5.01)$. Now, should we conclude that the true value is $0$? I would not feel comfortable saying that, because the CI is very wide, and there are many, large non-zero values that we might reasonably suspect are consistent with our data. So let's say we gather much, much more data, and now our observed mean difference is $0.01$, but the 95% CI is $(0.005,\ 0.015)$. The observed mean difference has stayed the same (which would be amazing if it really happened), but the confidence interval not excludes the null value. Of course, this is just a thought experiment, but it should make the basic ideas clear. We can never prove that the true value is any particular point value; we can only (possibly) disprove that it is some point value. In statistical hypothesis testing, the fact that the p-value is > 0.05 &/or the 95% CI includes zero means that we are not sure if the null hypothesis is true.

So what can you do? In this situation, you use equivalence testing. (You might want to read through some of our threads on this topic by clicking on the equivalence tag.) The typical strategy is to use the two one sided tests approach. Very briefly, you select an interval within which you would consider that the true mean difference might as well be $0$ for all you could care, then you perform a one-sided test to determine if the observed value is less than the upper bound of that interval, and another one-sided test to see if it is greater than the lower bound. If both of these tests are significant, then you have rejected the hypothesis that the true value is outside the interval you care about. If one (or both) are non-significant, you fail to reject the hypothesis that the true value is outside the interval. For example, suppose anything within the interval $(-0.02,\ 0.02)$ is so close to zero that you think it is essentially the same as zero for your purposes, so you use that as your substantive hypothesis. Now imagine that you get the first result described above. Although $0.01$ falls within that interval, you would not be able to reject the null hypothesis on either one-sided t-test, so you would fail to reject the null hypothesis. On the other hand, imagine that you got the second result described above. Now you find that the observed value falls within the designated interval, and it can be shown to be both less than the upper bound and greater than the lower bound, so you can reject the null. (It is worth noting that you can reject both the hypothesis that the true value is $0$, and the hypothesis that the true value lies outside of the interval $(-0.02,\ 0.02)$, which may seem perplexing at first, but is fully consistent with the logic of hypothesis testing.)

Traditionally, the null hypothesis is a point value. (It is typically $0$, but can in fact be any point value.) The alternative hypothesis is that the true value is any value other than the null value. Because a continuous variable (such as a mean difference) can take on a value which is indefinitely close to the null value but still not quite equal and thus make the null hypothesis false, a traditional point null hypothesis cannot be proven.

Imagine your null hypothesis is $0$, and the mean difference you observe is $0.01$. Is it reasonable to assume the null hypothesis is true? You don't know yet; it would be helpful to know what our confidence interval looks like. Let's say that your 95% confidence interval is $(-4.99,\ 5.01)$. Now, should we conclude that the true value is $0$? I would not feel comfortable saying that, because the CI is very wide, and there are many, large non-zero values that we might reasonably suspect are consistent with our data. So let's say we gather much, much more data, and now our observed mean difference is $0.01$, but the 95% CI is $(0.005,\ 0.015)$. The observed mean difference has stayed the same (which would be amazing if it really happened), but the confidence interval now excludes the null value. Of course, this is just a thought experiment, but it should make the basic ideas clear. We can never prove that the true value is any particular point value; we can only (possibly) disprove that it is some point value. In statistical hypothesis testing, the fact that the p-value is > 0.05 (and that the 95% CI includes zero) means that we are not sure if the null hypothesis is true.

So what can you do? In this situation, you use equivalence testing. (You might want to read through some of our threads on this topic by clicking on the equivalence tag.) The typical strategy is to use the two one sided tests approach. Very briefly, you select an interval within which you would consider that the true mean difference might as well be $0$ for all you could care, then you perform a one-sided test to determine if the observed value is less than the upper bound of that interval, and another one-sided test to see if it is greater than the lower bound. If both of these tests are significant, then you have rejected the hypothesis that the true value is outside the interval you care about. If one (or both) are non-significant, you fail to reject the hypothesis that the true value is outside the interval.

For example, suppose anything within the interval $(-0.02,\ 0.02)$ is so close to zero that you think it is essentially the same as zero for your purposes, so you use that as your substantive hypothesis. Now imagine that you get the first result described above. Although $0.01$ falls within that interval, you would not be able to reject the null hypothesis on either one-sided t-test, so you would fail to reject the null hypothesis. On the other hand, imagine that you got the second result described above. Now you find that the observed value falls within the designated interval, and it can be shown to be both less than the upper bound and greater than the lower bound, so you can reject the null. (It is worth noting that you can reject both the hypothesis that the true value is $0$, and the hypothesis that the true value lies outside of the interval $(-0.02,\ 0.02)$, which may seem perplexing at first, but is fully consistent with the logic of hypothesis testing.)

Imagine your null hypothesis is $0$, and the mean difference you observe is $0.01$. Is it reasonable to assume the null hypothesis is true? You don't know yet; it would be helpful to know what our confidence interval looks like. Let's say that your 95% confidence interval is $(-4.99,\ 5.01)$. Now, should we conclude that the true value is $0$? I would not feel comfortable saying that, because the CI is very wide, and there are many, large non-zero values that we might reasonably suspect are consistent with our data. So let's say we gather much, much more data, and now our observed mean difference is $0.01$, but the 95% CI is $(0.005,\ 0.015)$. The observed mean difference has stayed the same (which would be amazing if it really happened), but the confidence interval not excludes the null value. Of course, this is just a though experiment, but it should make the basic ideas clear.

So what can you do? In this situation, you use equivalence testing. (You might want to read through some of our threads on this topic by clicking on the equivalence tag.) The typical strategy is to use the two one sided tests approach.

Imagine your null hypothesis is $0$, and the mean difference you observe is $0.01$. Is it reasonable to assume the null hypothesis is true? You don't know yet; it would be helpful to know what our confidence interval looks like. Let's say that your 95% confidence interval is $(-4.99,\ 5.01)$. Now, should we conclude that the true value is $0$? I would not feel comfortable saying that, because the CI is very wide, and there are many, large non-zero values that we might reasonably suspect are consistent with our data. So let's say we gather much, much more data, and now our observed mean difference is $0.01$, but the 95% CI is $(0.005,\ 0.015)$. The observed mean difference has stayed the same (which would be amazing if it really happened), but the confidence interval not excludes the null value. Of course, this is just a thought experiment, but it should make the basic ideas clear. We can never prove that the true value is any particular point value; we can only (possibly) disprove that it is some point value. In statistical hypothesis testing, the fact that the p-value is > 0.05 &/or the 95% CI includes zero means that we are not sure if the null hypothesis is true.

So what can you do? In this situation, you use equivalence testing. (You might want to read through some of our threads on this topic by clicking on the equivalence tag.) The typical strategy is to use the two one sided tests approach. Very briefly, you select an interval within which you would consider that the true mean difference might as well be $0$ for all you could care, then you perform a one-sided test to determine if the observed value is less than the upper bound of that interval, and another one-sided test to see if it is greater than the lower bound. If both of these tests are significant, then you have rejected the hypothesis that the true value is outside the interval you care about. If one (or both) are non-significant, you fail to reject the hypothesis that the true value is outside the interval. For example, suppose anything within the interval $(-0.02,\ 0.02)$ is so close to zero that you think it is essentially the same as zero for your purposes, so you use that as your substantive hypothesis. Now imagine that you get the first result described above. Although $0.01$ falls within that interval, you would not be able to reject the null hypothesis on either one-sided t-test, so you would fail to reject the null hypothesis. On the other hand, imagine that you got the second result described above. Now you find that the observed value falls within the designated interval, and it can be shown to be both less than the upper bound and greater than the lower bound, so you can reject the null. (It is worth noting that you can reject both the hypothesis that the true value is $0$, and the hypothesis that the true value lies outside of the interval $(-0.02,\ 0.02)$, which may seem perplexing at first, but is fully consistent with the logic of hypothesis testing.)