Why is a one-sided test ever used when a sampling distribution is normal?

Question

If I understand correctly, sampling distributions of means or of proportions can be assumed to be normally distributed if certain conditions are met. However, if I also understand basic hypothesis testing correctly, a one proportion z test allows one to posit a one sided alternative hypothesis, e.g.:

• $H_0: p \le 0.5$
• $H_A: p > 0.5$

This suggests to me that we wouldn't consider a value more than $3\, \text{SDs}$ below $p$ to be surprising, yet that seems to contradict the very definition of a normal distribution.

I suppose this doesn't matter if the critical value is calculated the same way regardless of whether you're doing a one-sided or two-sided test in such a scenario, but if I'm understanding correctly, the critical value for the same level of confidence comes out to a different value with each type of test.

I must be misunderstanding some part of this, otherwise I just don't understand why a one-sided test is ever allowed for a normal distribution.

EDIT:

Perhaps I'm not asking this question in a way that's comprehensible. I'm sure I have imprecise vocabulary at this point, so I'm going to try using a picture:

What I'm trying to get at is that a sampling distribution of proportions is supposed to be normally distributed according to the central limit theorem, yet, to me, a one-tailed z test seems to imply that it's distributed as shown in the bottom figure, because a value of -1.96 or lower is not seen as unexpected given the null hypothesis when performing a one-tailed z test for a sampling distribution of proportions. I must be getting some part of this wrong, but no matter how many descriptions I read or videos I watch or clarifying questions I ask, I can't seem to identify why a one-tailed z test for a sampling distribution of proportions is still drawn as a normal distribution. It would only make sense to me to draw it as a normal distribution if we were keeping the critical value at 1.96 and simply ignoring results that are less than -1.96, while still maintaining that they're unexpected, i.e. cutting the top figure in half.

Answer 1

You use a one sided test when you have a one sided hypothesis. The shape of the distribution of whatever it is you are testing has nothing to do with it. You wrote:

This suggests to me that we wouldn't consider a value more than 3SDs below p to be surprising,

but p values are not about "surprisingness", they are about statistical significance. It might be completely expected to have a very low p value. Statistical significance means that, if, in the population from which your sample was randomly drawn the null hypothesis was true, you would be unlikely to get a test statistic at least as extreme as the one you got in a sample the size of the one you have.

As for an example of a case where we want a one sided test for a proportion, suppose we suspect that a coin is weighted to favor heads. We might then test if the proportion of heads is > 0.5.

Answer 2

Equivalence tests can be a super-useful purpose for one-sided tests

If you want to provide evidence that two quantities are equivalent, rather than evidence that two quantities are different, you can use the two one-sided tests approach (TOST):

$H^{-}_{0}: |\mu_{1}-\mu_{2}|\ge \Delta$
$H^{-}_{A}: |\mu_{1}-\mu_{2}|< \Delta$

This is the general form of the TOST null hypothesis (where $\Delta$ is a researcher choice meaning "the smallest difference between two quantities that I care about"), which translates into two specific one-sided nulls:

$H^{-}_{01}: \mu_{1}-\mu_{2}\ge \Delta$
$H^{-}_{A1}: \mu_{1}-\mu_{2}< \Delta;$

or

$H^{-}_{02}: \mu_{1}-\mu_{2}\le -\Delta$
$H^{-}_{A2}: \mu_{1}-\mu_{2}> -\Delta;$

If you reject $H^{-}_{01}$ then you infer that $\mu_{1}-\mu_{2}$ must be less than $\Delta$. If you reject $H^{-}_{02}$ then you infer that $\mu_{1}-\mu_{2}$ must be greater than $-\Delta$. If your reject both $H^{-}_{01}$ and $H^{-}_{02}$, then you must conclude that $-\Delta < \mu_{1}-\mu_{2} < \Delta$, or in plain language that the two quantities are equivalent with the range $\pm\Delta$.

Non-inferiority and non-superiority tests are another super-useful purpose for one-sided tests: If you are bringing a drug to market, as a producer you do not want to show to regulators that your drug is better than existing drugs (why should you be held to a different standard?), but that your drug performs no worse than the best available existing drugs, within a margin of relevance. The non-inferiority null and alternative hypotheses employ one-sided tests:

$H_{0}: \mu_{1} \le \mu_{0} -\Delta$
$H_{A}: \mu_{1} > \mu_{0} -\Delta$

The non-superiority test simply inverts these relationships:

$H_{0}: \mu_{1} \ge \mu_{0} +\Delta$
$H_{A}: \mu_{1} < \mu_{0} +\Delta$

A brief examination will reveal that the equivalence test's alternative hypothesis is the intersection of the alternative non-inferiority and non-superiority alternative hypotheses

Answer 3

To add to Peter's excellent answer, the formulation of the alternative hypothesis depends on the research question.

As an example, if p denotes the proportion of people in California who watch TV in the evening, you could set a directional alternative hypothesis as either Ha: p > 0.5 or Ha: p < 0.5 depending on whether you expect that proportion to be greater than 0.5 or smaller than 0.5. If you have no such expectation (i.e., you believe the proportion could in principle be either smaller than 0.5 or greater than 0.5), you would set a non-directional alternative hypothesis as Ha: p != 0.5 where != stands for "not equal to".

Once you know what alternative hypothesis you wish to test, the null hypothesis Ho is easy to derive. For Ha: p > 0.5, the null hypothesis can be set up as Ho: p <= 0.5, so that Ho and Ha are mutually exhaustive, etc.

With the hypotheses in place, the observed value of the test statistic can be evaluated from the data. This observed value will then be used in conjunction with information on:

(1) the sampling distribution of the test statistic when Ho is assumed to be true and

(2) the alternative hypothesis

in order to derive a p-value associated with testing Ho against Ha.

For all three types of alternative hypotheses stated above, the observed value of the test statistic and the shape of the sampling distribution will be the same. But because the alternative hypotheses are different, the corresponding p-values will be different. So the nature of the alternative hypothesis drives what kind of p-value you will get.

Answer 4

The attached image may help. If we are looking for evidence against the null hypothesis H0: p <= 0.5 when testing it against the alternative hypothesis Ha: p > 0.5, it makes sense to compute the p-value by looking under the right tail of the sampling distribution of the test statistic (which was derived under the assumption that H0 is true). That tail encapsulates outcomes for our test which would be highly unlikely were the null hypothesis really true. Indeed, if the null hypothesis were true, we would not expect highly positive values for the test statistic. So if we find a highly positive value for the observed test statistic, we suspect that our assumption that H0 is true is likely unfounded (i.e., not supported by the data).

I think some of the questions in the comment thread need to be formulated more clearly to receive proper answers. Perhaps start a new thread with a clearer set of questions rather than have the comments go on forever?

Answer 5

I finally figured out why a one-tailed test is not contradicting the probabilities that a normal distribution implies: because an $\alpha$ of 0.05 for a one-tailed test is equivalent to an $\alpha$ of 0.10 for a two-tailed test if we were to ignore one side of the distribution in the latter, and since we are ignoring one side or the other of the distribution in a one-tailed test, it still makes sense to talk about about a P-value > 0.05 or < 0.05 because only 5% of the samples taken from that population are expected to fall in that region of the distribution.

I'm still not sure why a two-tailed test with a higher $\alpha$ isn't used instead, but that's another question which I suppose comes down to logistical concerns (e.g. the sample doesn't have to be as large for a one-tailed test?).

(This discussion is what helped me understand the part of the puzzle that I was missing: http://spssx-discussion.1045642.n5.nabble.com/1-tailed-test-and-alpha-td1071625.html)

Answer 6

Thank you so much joshisanonymous for asking this question! I think it is a really good question and I was wondering about it myself as well. Personally I find it difficult to fully understand statistics conceptually, and often I also find it hard to find satisfying answers on the internet. That's why I usually just come up with answers myself... So be aware: I am no expert!

As opposed to Peter I do think that the shape of the distribution of what you are testing has a lot to do with your test and hypothesis, no matter if it is one-sided or not. The whole idea of a test in my perception is assuming a certain underlying distribution and computing the likeliness of your sample data under that assumption.

My conclusion: I think that a difference between a one-sided test and a two-sided test is that the probability of committing a type 1 error in a one-sided test is at most alpha, since you take into account the collection of all distributions (of a particular shape) with mean smaller than or equal to 0.5 in this case, whereas the probability of ending up in the tails for a two-sided test is alpha, since you only look at one distribution.

Below I look at the sampling distribution of the mean, but I did not scale the distribution to t-distribution or standard normal as you did in your picture. I hope it helps!