1
$\begingroup$

If I understand correctly, sampling distributions of means or of proportions can be assumed to be normally distributed if certain conditions are met. However, if I also understand basic hypothesis testing correctly, a one proportion z test allows one to posit a one sided alternative hypothesis, e.g.:

  • $H_0: p \le 0.5$
  • $H_A: p > 0.5$

This suggests to me that we wouldn't consider a value more than $3\, \text{SDs}$ below $p$ to be surprising, yet that seems to contradict the very definition of a normal distribution.

I suppose this doesn't matter if the critical value is calculated the same way regardless of whether you're doing a one-sided or two-sided test in such a scenario, but if I'm understanding correctly, the critical value for the same level of confidence comes out to a different value with each type of test.

I must be misunderstanding some part of this, otherwise I just don't understand why a one-sided test is ever allowed for a normal distribution.

EDIT:

Perhaps I'm not asking this question in a way that's comprehensible. I'm sure I have imprecise vocabulary at this point, so I'm going to try using a picture:

enter image description here

What I'm trying to get at is that a sampling distribution of proportions is supposed to be normally distributed according to the central limit theorem, yet, to me, a one-tailed z test seems to imply that it's distributed as shown in the bottom figure, because a value of -1.96 or lower is not seen as unexpected given the null hypothesis when performing a one-tailed z test for a sampling distribution of proportions. I must be getting some part of this wrong, but no matter how many descriptions I read or videos I watch or clarifying questions I ask, I can't seem to identify why a one-tailed z test for a sampling distribution of proportions is still drawn as a normal distribution. It would only make sense to me to draw it as a normal distribution if we were keeping the critical value at 1.96 and simply ignoring results that are less than -1.96, while still maintaining that they're unexpected, i.e. cutting the top figure in half.

$\endgroup$
  • $\begingroup$ It will be helpful if you move your answer to your question into another answer rather than as an edit. It's already really hard for me to grasp what this question is about, and the edits seem like a whole other question. You can select your answer as the correct one if it is "solved" as you say. Following the site conventions will make this overall more accessible to everyone. $\endgroup$ – AdamO Mar 20 '18 at 13:46
  • $\begingroup$ @AdamO Good advice. Done. $\endgroup$ – joshisanonymous Mar 20 '18 at 15:14
  • $\begingroup$ What kind of distribution did your draw in your lower plot? It does not seem to integrate to one. $\endgroup$ – tomka Feb 14 at 15:15
  • $\begingroup$ @tomka It's supposed to be a normal distribution in a case where you're pretending that the lower tail doesn't exist and so can draw that side any way you want. In other words, I was just trying to highlight that it seems like that tail is being ignored. A better way would have been to simply cover everything to the left of the mean, I think. $\endgroup$ – joshisanonymous Feb 19 at 16:36
6
$\begingroup$

You use a one sided test when you have a one sided hypothesis. The shape of the distribution of whatever it is you are testing has nothing to do with it. You wrote:

This suggests to me that we wouldn't consider a value more than 3SDs below p to be surprising,

but p values are not about "surprisingness", they are about statistical significance. It might be completely expected to have a very low p value. Statistical significance means that, if, in the population from which your sample was randomly drawn the null hypothesis was true, you would be unlikely to get a test statistic at least as extreme as the one you got in a sample the size of the one you have.

As for an example of a case where we want a one sided test for a proportion, suppose we suspect that a coin is weighted to favor heads. We might then test if the proportion of heads is > 0.5.

$\endgroup$
  • $\begingroup$ This still doesn't make sense to me. You say that it's not about surprisingness, but you seem to be saying that it's not surprising for you if you didn't believe the assumption that the null hypothesis is true anyway. From the point of view of the null hypothesis, isn't a value more than 3 SDs in any direction from the mean when normally distributed still "surprising"? Don't "unlikely" and "extreme" suggest something surprising from the point of view of an assumed null? $\endgroup$ – joshisanonymous Mar 19 '18 at 12:04
  • 1
    $\begingroup$ It would be surprising if the null was true, but that's not the point. The point is to test a hypothesis. If your hypothesis is one-tailed, your test is one-tailed. Then you are not even looking for the other tail. $\endgroup$ – Peter Flom - Reinstate Monica Mar 19 '18 at 12:06
  • $\begingroup$ But that is the point, isn't it? What I'm trying to get at with my question is that it seems like when doing a one-tailed test, the statistician is deciding that extreme values to one side are actually quite likely, but isn't it the distribution that decides what's likely? It wouldn't matter if the critical value came out the same for both one-tailed and two-tailed tests, but my understanding is that it changes based on the type of test you do because, with a one-tailed test, you're saying that a value 3 SDs or more below (or above) the mean of a normal distribution is perfectly likely. $\endgroup$ – joshisanonymous Mar 19 '18 at 12:16
  • 1
    $\begingroup$ A one tailed test says that extreme values on one side are of no importance and not interesting and, therefore, doesn't look for them. $\endgroup$ – Peter Flom - Reinstate Monica Mar 19 '18 at 12:23
  • 2
    $\begingroup$ How it seems to you is simply wrong. Sorry. It says nothing about what is expected given sampling variability it says things about what is expected given the null hypothesis. If you change the null, you change what is expected. $\endgroup$ – Peter Flom - Reinstate Monica Mar 19 '18 at 19:54
3
$\begingroup$

Equivalence tests can be a super-useful purpose for one-sided tests

If you want to provide evidence that two quantities are equivalent, rather than evidence that two quantities are different, you can use the two one-sided tests approach (TOST):

$H^{-}_{0}: |\mu_{1}-\mu_{2}|\ge \Delta$
$H^{-}_{A}: |\mu_{1}-\mu_{2}|< \Delta$

This is the general form of the TOST null hypothesis (where $\Delta$ is a researcher choice meaning "the smallest difference between two quantities that I care about"), which translates into two specific one-sided nulls:

$H^{-}_{01}: \mu_{1}-\mu_{2}\ge \Delta$
$H^{-}_{A1}: \mu_{1}-\mu_{2}< \Delta;$

or

$H^{-}_{02}: \mu_{1}-\mu_{2}\le -\Delta$
$H^{-}_{A2}: \mu_{1}-\mu_{2}> -\Delta;$

If you reject $H^{-}_{01}$ then you infer that $\mu_{1}-\mu_{2}$ must be less than $\Delta$. If you reject $H^{-}_{02}$ then you infer that $\mu_{1}-\mu_{2}$ must be greater than $-\Delta$. If your reject both $H^{-}_{01}$ and $H^{-}_{02}$, then you must conclude that $-\Delta < \mu_{1}-\mu_{2} < \Delta$, or in plain language that the two quantities are equivalent with the range $\pm\Delta$.

Non-inferiority and non-superiority tests are another super-useful purpose for one-sided tests: If you are bringing a drug to market, as a producer you do not want to show to regulators that your drug is better than existing drugs (why should you be held to a different standard?), but that your drug performs no worse than the best available existing drugs, within a margin of relevance. The non-inferiority null and alternative hypotheses employ one-sided tests:

$H_{0}: \mu_{1} \le \mu_{0} -\Delta$
$H_{A}: \mu_{1} > \mu_{0} -\Delta$

The non-superiority test simply inverts these relationships:

$H_{0}: \mu_{1} \ge \mu_{0} +\Delta$
$H_{A}: \mu_{1} < \mu_{0} +\Delta$

A brief examination will reveal that the equivalence test's alternative hypothesis is the intersection of the alternative non-inferiority and non-superiority alternative hypotheses

$\endgroup$
  • 2
    $\begingroup$ +1 for mention of non-inferiority tests. If I presume that my lab has created a biosimilar to, say, lovastatin, I must do a trial to prove it. Since I have the burden of proof. my null is that my drug is worse than lovastatin (harm). I don't care how much worse it is, I'll stop the trial as soon as there's any evidence of harm. $\endgroup$ – AdamO Mar 20 '18 at 13:44
2
$\begingroup$

To add to Peter's excellent answer, the formulation of the alternative hypothesis depends on the research question.

As an example, if p denotes the proportion of people in California who watch TV in the evening, you could set a directional alternative hypothesis as either Ha: p > 0.5 or Ha: p < 0.5 depending on whether you expect that proportion to be greater than 0.5 or smaller than 0.5. If you have no such expectation (i.e., you believe the proportion could in principle be either smaller than 0.5 or greater than 0.5), you would set a non-directional alternative hypothesis as Ha: p != 0.5 where != stands for "not equal to".

Once you know what alternative hypothesis you wish to test, the null hypothesis Ho is easy to derive. For Ha: p > 0.5, the null hypothesis can be set up as Ho: p <= 0.5, so that Ho and Ha are mutually exhaustive, etc.

With the hypotheses in place, the observed value of the test statistic can be evaluated from the data. This observed value will then be used in conjunction with information on:

(1) the sampling distribution of the test statistic when Ho is assumed to be true and

(2) the alternative hypothesis

in order to derive a p-value associated with testing Ho against Ha.

For all three types of alternative hypotheses stated above, the observed value of the test statistic and the shape of the sampling distribution will be the same. But because the alternative hypotheses are different, the corresponding p-values will be different. So the nature of the alternative hypothesis drives what kind of p-value you will get.

$\endgroup$
1
$\begingroup$

enter image description here

The attached image may help. If we are looking for evidence against the null hypothesis H0: p <= 0.5 when testing it against the alternative hypothesis Ha: p > 0.5, it makes sense to compute the p-value by looking under the right tail of the sampling distribution of the test statistic (which was derived under the assumption that H0 is true). That tail encapsulates outcomes for our test which would be highly unlikely were the null hypothesis really true. Indeed, if the null hypothesis were true, we would not expect highly positive values for the test statistic. So if we find a highly positive value for the observed test statistic, we suspect that our assumption that H0 is true is likely unfounded (i.e., not supported by the data).

I think some of the questions in the comment thread need to be formulated more clearly to receive proper answers. Perhaps start a new thread with a clearer set of questions rather than have the comments go on forever?

$\endgroup$
  • 1
    $\begingroup$ re "new thread": This site has been active almost eight years and has accumulated over 100,000 (unclosed, undeleted) questions. It would be surprising if this basic issue hadn't been discussed hundreds, if not thousands of times, in all kinds of settings. Although our reflex is to supply an answer, it's often better to begin by looking for what's already out there. stats.stackexchange.com/questions/tagged/… is one search that will turn up useful material. The top-voted hit addresses many common questions about p-values from a wide variety of points of view. $\endgroup$ – whuber Mar 19 '18 at 20:14
0
$\begingroup$

I finally figured out why a one-tailed test is not contradicting the probabilities that a normal distribution implies: because an $\alpha$ of 0.05 for a one-tailed test is equivalent to an $\alpha$ of 0.10 for a two-tailed test if we were to ignore one side of the distribution in the latter, and since we are ignoring one side or the other of the distribution in a one-tailed test, it still makes sense to talk about about a P-value > 0.05 or < 0.05 because only 5% of the samples taken from that population are expected to fall in that region of the distribution.

I'm still not sure why a two-tailed test with a higher $\alpha$ isn't used instead, but that's another question which I suppose comes down to logistical concerns (e.g. the sample doesn't have to be as large for a one-tailed test?).

(This discussion is what helped me understand the part of the puzzle that I was missing: http://spssx-discussion.1045642.n5.nabble.com/1-tailed-test-and-alpha-td1071625.html)

$\endgroup$
0
$\begingroup$

Thank you so much joshisanonymous for asking this question! I think it is a really good question and I was wondering about it myself as well. Personally I find it difficult to fully understand statistics conceptually, and often I also find it hard to find satisfying answers on the internet. That's why I usually just come up with answers myself... So be aware: I am no expert!

As opposed to Peter I do think that the shape of the distribution of what you are testing has a lot to do with your test and hypothesis, no matter if it is one-sided or not. The whole idea of a test in my perception is assuming a certain underlying distribution and computing the likeliness of your sample data under that assumption.

My conclusion: I think that a difference between a one-sided test and a two-sided test is that the probability of committing a type 1 error in a one-sided test is at most alpha, since you take into account the collection of all distributions (of a particular shape) with mean smaller than or equal to 0.5 in this case, whereas the probability of ending up in the tails for a two-sided test is alpha, since you only look at one distribution.

Below I look at the sampling distribution of the mean, but I did not scale the distribution to t-distribution or standard normal as you did in your picture. I hope it helps!

One-sided test: possible sampling distributions under null-hypothesis

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.