What happens when there's a one-tailed hypothesis test and the results are in the other tail?

Question

For example, let's say I'm using a one-tailed t-test to see if a box of Lucky Charms cereal has more than 100 marshmallows. I know the consequences of using a one-tailed t-test and how it wouldn't be appropriate here. However, if my t-value is -2 and there are way less marshmallows than my hypothesis, when doing the one-tailed test and I use a t-value calculator like this, it doesn't even ask which tail I want to use. So if I put in -2 and maybe a df of 9, the P-value is 0.038, which is significant. So when Wikipedia says "values in the other direction are considered not significant", what does this mean exactly? How do I correct my 0.038 to make it representative of a one-tailed t-test on the other tail - I was thinking would you say 1-0.038 to get a P-value of .962? Thanks so much.

Answer 1

For a two-sided test, the $p$-value is $$ p(\hat{z}) = 2(1- \Phi(|\hat{z}|)), $$ where $\hat{z}$ is the $t$-statistic and $\Phi$ is the standard normal cdf (assuming our test is standard normal under the null, of course - if your work with the $t$-distribution, just replace all instances of $\Phi$ with the appropriate cdf of the $t$-distribution).

To see this, note that the test based on $\hat{z}$ rejects if $$|\hat{z}| > c_\alpha$$ For a test at the 5%-level, $\alpha$ would be 0.975, so that $c_\alpha$ is the 97.5%-quantile of the standard normal distribution.

The above display is equivalent to $$\Phi(|\hat{z}|) > \Phi(c_\alpha),$$ because $\Phi$ is strictly increasing.

Further, recall the definition of the critical value $$ \Phi(c_\alpha)=1-\alpha/2 $$ The smallest value of $\alpha$ for which the inequality holds is thus obtained by solving the equation$$\Phi(|\hat{z}|) =1-\alpha/2$$ for $\alpha$, which gives $2(1- \Phi(|\hat{z}|))$.

For a one-sided (right-tailed) test, the test based on $\hat{z}$ rejects if $$\hat{z} > c_{2\alpha}$$ So we now solve $$ \Phi(\hat{z}) =1-2\alpha/2 $$ for $\alpha$ to obtain $$1-\Phi(\hat{z})$$ as the definition of the $p$-value for a right-tailed test.

In the figures, I display the $p$-value for a two-sided as well as for a right-tailed test given a test statistic $t=1.8$. In the first case, we also need the yellow area to the left of $-t$ (next to that right of $t$), because very negative values of $t$ are also evidence against a two-sided $H_0$. By symmetry of the normal, we see however that these two areas are the same, whence we can simply compute $2(1- \Phi(|\hat{z}|))$.

In the right-tailed test, only large values are evidence against $H_0$, so we only need the yellow area to the right of $t$.

In your example,

> 1-pnorm(-2)
[1] 0.9772499

EDIT:

With the OP's t-distribution with 9 dfs (cf. comments), we get

> 1-pt(-2,9)
[1] 0.9617236