While studying for my stats course, I was trying to understand the difference between one-tailed and two-tailed hypothesis tests. Specifically, why does the one-tailed test reject the null while the two-tailed one does not?
An example:
$\begingroup$
$\endgroup$
1
-
$\begingroup$ Note that you are only rejecting at a specified significance level. You can still reject both hypothesis by raising the significance level to 10%. you would also fail to reject both if you lowered the significance level to 1%. $\endgroup$– probabilityislogicMar 15, 2012 at 12:31
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
5
A two tailed test tests for a difference in either direction. Thus the P value would be the area under the t distribution to the right of t=1.92 PLUS the area under the distribution to the left of t=-1.92. That's twice as much area as the one-tailed test and so the P value is twice as large.
If you use a one tailed test you gain power, but at the potential cost of having to ignore a difference that is in the opposite direction to that hypothesised before the data were obtained. If you got the data before you formalised and recorded the hypothesis you really should use a two tailed test. Similarly, if you would be interested in an effect in either direction you use a two tailed test. In fact, you may wish to use a two-tailed test as your default approach and only use a one-tailed test in the unusual case where an effect can only exist in one direction.
answered Mar 15, 2012 at 3:34
-
$\begingroup$ Thank you for your comment, Michael. Here is what I don't understand: how can the area under the curve be twice as large for the two-tailed test? Shouldn't P be the same in both cases, since alpha= 0.05? $\endgroup$– Lu CiMar 15, 2012 at 3:52
-
$\begingroup$ alpha, in your question, is just your cutoff for making a decision about what p-means (reject null or not). So, it doesn't influence what the value of p is. $\endgroup$– JohnMar 15, 2012 at 4:52
-
$\begingroup$ A bit nit-picky but the notion that you need to choose the hypothesis before seeing the data is not necessary. You can do two one sided tests. You will always reject the direction not favoured by the data. Thus it makes sense to go with the one sided test which is favoured by the data. $\endgroup$ Mar 15, 2012 at 13:31
-
1$\begingroup$ @probabilityislogic -I agree entirely, except for when one wants the alpha value to reflect the actual long term false positive error rate (i.e. one wants to use the Neyman-Pearson approach). If you use a one-sided test and decide on the direction of change to test for on the basis of the observed effect then you get exactly twice as many false positive results as your alpha level would imply. $\endgroup$ Mar 16, 2012 at 0:26
-
$\begingroup$ It is only the long term error rate for someone who keeps ignoring previous data. This is not a good way to achieve good long run performance. the error rate given by significance is over all the data sets we could have observed once. $\endgroup$ Mar 16, 2012 at 8:52
$\begingroup$
$\endgroup$
The area under the curve is not twice as large for a two tailed test: For a two tailed test with critical p= .05, you are testing how often the observed data could be drawn from lower or upper 2.5% of a null distribution (.05 in total). With a 1-tailed test, you are testing how often the data would come from the extreme 5% tail of one (pre-specified) tail.
Partly the answer to your question is one of practice: Most researchers view experiments reporting 1-tailed tests as unlikely to replicate (i.e., they assume the researcher chose this to get their stats to be "significant").
There are however valid use cases. If you know that any result in the reverse direction is impossible under the theory being tested, then, as a previous comment noted, you can specify this ahead of time and conduct a 1-tailed test. Most people, again, would still view this circumspectly.
$\begingroup$
$\endgroup$
The reason for the difference is "hidden" in the test statistic used for each test. note that for hypothesis testing you choose a statistic (ie a function of the data) to base the test on. call this statistic $S(D)$. You also require a rejection region $RR$ such that if the statistic is in the region, we reject the null. now the significance level is calculated as the probability that the statistic is in the rejection region, when the null is true.
Now for the two sided test the test statistic is $S(D)=|t|$ with rejection region $|t|>t_{0}$ where $t_{0}$ is chosen to achieve significance $\alpha$ which is 5% in your case. for the one sided test the test statistic is $S(D)=t$ and the rejection region is $t>t_{1}$ for suitably chosen $t_{1}$. now we will always have $Pr(|t|>t_{0}|H_{0})\geq Pr(t>t_{0}|H_{0})$. so in order to achieve same significance we must have $t_{0}\geq t_{1}$.
This leads to the question: why use different test statistics? The reason is that the alternatives are different and so the power of each test statistic is different. Specifically the power of each test is reduced (provided we use same significance) if we use the test statistic and rejection region from the other test.
answered Mar 15, 2012 at 13:17