4
$\begingroup$

I was wondering if anyone could help explain to me why someone would use a two tailed hypothesis test? Let's say I am doing a t-test, presumably I know which mean was greater than the other and I am testing if that difference is statistically significant (i.e. a one tailed test). Given the formula for computing the t-statistic, whether the value is positive or negative just depends on which group I label as group 1 or 2, no? Are there applications for a two tailed test with a different hypothesis testing method? Or am I missing something?

Edit: To clarify, I am really wondering why you wouldn't just identify which group gives greater results, and use a one-tailed test. Because if there is a difference, that difference is going to go one way or the other, and then you can test accordingly.

Improve this question
$\endgroup$
1
  • 4
    $\begingroup$ You should never design your statistical analysis after the data is collected. With this in mind it is clear that in the majority of cases you don't know in advance the relative order of statistics of group 1 and 2. This is why we have two tailed tests. $\endgroup$
    – Mur1lo
    Commented Apr 17, 2020 at 2:11

3 Answers 3

Reset to default
9
$\begingroup$

Actually, in the context of the test of mean differences, it tends to be the other way around --- it is almost never appropriate to use a one-sided test. The reason for this is that we need to specify our objects of inference (e.g., hypothesis tests, confidence intervals, etc.) prior to seeing the data, or we will induce bias in these objects. When seeking to make inference about two unknown quantities, it is generally best not to assume that the direction of interest is known a priori, and so it is usually best to test for a difference rather than a directional difference. Others will argue that it is legitimate to use a one-sided test when you have specified a direction of interest a priori, but I am sceptical even in this case. I would counsel that you should either avoid classical hypothesis testing altogether (e.g., using a confidence interval instead) or use a two-sided hypothesis test, even if you are interested in a relationship with a specified direction.

In regard to this issue, it is worth noting that classical hypothesis tests have some unusual (and not very helpful) properties when you compare across different tests. One of their properties is that, for a symmetric test, the p-value of the two-sided test is twice as high as the p-value for the one-sided test when data is in the relevant tail. This means that if you do a one-sided test for a disparity in the direction of the data, the p-value will be half the size of a two-sided test. So, if you correctly guess the direction of the trend a priori, the result of using the one-sided test is that you see evidence that looks twice as strong for the more specific hypothesis! This property of classical hypothesis tests gives good reason to avoid one-sided tests.

In any case, whether or not you agree with my view here, what you are proposing is definitely a bad idea. If you identify the direction of the test from the observed data, and then perform a one-sided test in the identified direction, you will bias your test towards rejection of the null hypothesis.

Improve this answer
$\endgroup$
5
  • $\begingroup$ You mentioned 'classical hypothesis tests,' is there a circumstance with different tests where this problem doesn't exist when doing a one tailed test? Is this problem just for tests where the test statistical is distributed symmetrically? And if this is the case, doesn't that make symmetrical tests less likely to give you statistically significant results inherantly, making a one tailed test closer to that of an assymetrical test? $\endgroup$
    – ajax2112
    Commented Apr 17, 2020 at 10:11
  • $\begingroup$ The same basic problem exists with non-symmetric tests in the classical framework (as opposed to say, the Bayesian framework), but the p-value is not exactly halved. $\endgroup$
    – Ben
    Commented Apr 17, 2020 at 10:19
  • $\begingroup$ –1 Non-inferiority, non-superiority, and TOST equivalence tests are all routine examples of one-sided tests. $\endgroup$
    – Alexis
    Commented Jun 30, 2021 at 0:47
  • 1
    $\begingroup$ @Alexis: Thanks for the explanation of your downvote; good of you to do. I am speaking here in the context of the original question, referring to a T-test for a mean difference, so I intend for my warnings to apply in that context. There are certainly some one-sided tests in other contexts that are fine (although I consider TOST to be essentially a kind of two-sided test, so I don't consider that to be a counter-example). I have edited the first sentence of my answer to be clearer on the restricted context I have in mind; hope that helps. $\endgroup$
    – Ben
    Commented Jun 30, 2021 at 2:17
  • 1
    $\begingroup$ @Ben Works for me. :) <3 $\endgroup$
    – Alexis
    Commented Jul 1, 2021 at 1:43
2
$\begingroup$

A one-tailed test is appropriate if you only want to test if there is a difference between your groups in a specific direction. You would use a two-tailed test if you want to determine if there is any difference between the two groups you're comparing.

As user Mur1lo says in their comment - you should never design your analysis after the data is collected. Therefore a two-tailed test is often more appropriate. A one-tailed test can only be justified if you have made a prediction prior to data collection about the direction of the difference, and you are completely uninterested in the possibility that the opposite outcome could be true.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I find “uninterested” to be an odd framing. I much prefer “if it would not change your behavior if the effect was in the unexpected direction.” In tech, I would not ship a flat or negative experiment, but I would definitely care that we built something that’s a bad user experience. $\endgroup$
    – dimitriy
    Commented Oct 15, 2023 at 4:20
  • $\begingroup$ When prior interest is in only one direction, a 2-tailed test is essentially a multiplicity adjustment to protect you against making a claim in the direction you don’t care about anyway. So it’s an excellent point to raise. Contrast that with Bayesian posterior probabilities which are directional, to gather evidence in favor of the assertion of interest. $\endgroup$
    – Frank Harrell
    Commented Oct 15, 2023 at 12:46
0
$\begingroup$

But the data is collected and p-value calculated REGARDLESS of your assumption, right? In other words, you don't have to make any assumptions prior to data collection. Let the data tell which null hypothesis is accepted or rejected. I see conflicting conclusions among the 3 scenarios: p>|t|, p>t and p<t

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.