# What if the P-Value is less than 0.05, but the test statistic is also less than the critical value?

Let's say you let the Null Hypothesis $$H_0$$ be that the mean volume of water in a bottle is some $$\mu$$, and the alternate hypothesis $$H_1$$ be that the mean volume is not $$\mu$$.

Let's assume you pick a 95% confidence interval, so $$\alpha = 0.05$$. That would give you a critical value (assuming a normal distribution) of 1.96.

Let's say the P-Value is less than 0.05, which would mean you reject $$H_0$$. But what if the test statistic is less than 1.96? In that case you would reject $$H_1$$, but you already rejected $$H_0$$ from the P-value. Does one take precedence over the other, or would this never happen?

• For example, for a right tailed test, p value is defined as $p = p(t) = \mathbb{P}(T > t | H_{0})$, where $T$ is the test statistic with known null distribution. When the distribution of the test statistic $T$ (when $H_0$ holds) is standard normal, it can be checked that $\mathbb{P}(T > 1.96) = 0.05$, and for this case $c_{0.05} = 1.96$ would be called a critical value. Naturally, the critical values may differ based on the null distribution of the test statistic (see, e.g., student's distribution). See here for more examples
– runr
Nov 26, 2020 at 15:02
• Nov 27, 2020 at 17:44

They’re synonyms. The critical value is the value the test statistic has to reach in order for your hypothesis test to reject at your chosen level (maybe $$0.05$$, maybe not). If you are not getting these to agree, there is a bug in your code.

You might be stumbling on the fact that confidence intervals and hypothesis tests (usually) are inverses of one another. The p-value, it turns out, is the $$\alpha$$-level for which a $$(1-\alpha)\%$$ confidence interval has the null value as one of the endpoint.

Hypothesis tests are always framed in terms of the null. In the case that the test statistic is less than the critical value, then the null fails to be rejected. When test statistic exceeds the critical value, we reject the null hypothesis.

To your point, the p value could be less than 0.05 and we could still have the test statistic be less than the critical value. This would mean our chosen $$\alpha$$ was smaller than 0.05, and would mean we would fail to reject the null.

• But what if we chose $\alpha$ to be 0.05, and the p value was less than $\alpha$ and the test statistic was less than the critical value? You're saying that this would not be possible, and the only explanation is that the $\alpha$ value chosen was actually smaller? I guess what I'm really asking is what if the two "methods" (p value vs $\alpha$ and test statistic vs critical value) of rejecting $H_0$ contradict each other? Nov 26, 2020 at 1:57
• They can never contradict one another. The p value is a function of the test statistic, the critical value is a function of alpha. Your "two methods" are two sides of the same coin. Nov 26, 2020 at 2:02
• There are no methods to choose from. There is only one procedure. Nov 26, 2020 at 2:42
• @JDL you can have a p value less than 0.05 and still fail to reject the null. In that case, the associated tests' alpha value is less than 0.05. For example, if I chose an alpha of 0.01, then a z test with such an alpha has a critical value of 2.57. If my test yielded a test statistic of 2.3 my p value would be 0.021. I would fail to reject the null because my p value is not smaller than 0.01 all the while having my p value smaller than 0.05 Nov 26, 2020 at 15:56
• @JDL your welcome to submit an answer you think is superior then. Nov 26, 2020 at 16:12