3
$\begingroup$

Let's say I'm doing a two-tailed hypothesis test at 5% significance level and get a test statistic that corresponds to a p-value of $0.03$. As it is two-tailed I double it and therefore, as $0.06 > 0.05$, I fail to reject the null hypothesis.

However, let's say now I'm checking for a result strictly greater than the mean so it's now a one-tailed test. At 5% significance level, do I now reject my null hypothesis as $0.03 < 0.05$ or as we're only doing a one-tailed test do I halve $0.05$ and check against that?

$\endgroup$
6
$\begingroup$

If you do a two-tailed test and computation gives you $p=0.03$, then $p<0.05$. The result is significant. If you do a one-tailed test, you will get a different result, depending on which tail you investigate. It will be either a lot larger or only half as big.

$\alpha=0.05$ is the usual convention, no matter whether you test one- ode two-tailed. You don't halve that (except maybe in Bonferroni-correction, which is not the topic here). Thus yes, sometimes a one-tailed test will give you a significant result where the two-tailed does not. However, this is not how things work: You have to always determine upfront, whether you consider a one- or a two-tailed test appropriate as you have to determine your $\alpha$-level upfront. Then you calculate the $p$-value for that test and there are no more degrees of freedom how to test or what to compare the $p$-value to. If you determine on the sidedness of your test depending on whether you like the result, this is not good scientific practice.

That being said, there is hardly ever a situation where it is appropriate to test one-tailed. In far most circumstances it would be worth communicating a significant result in both directions. If you test one-tailed, some of your audience will consider this a trick to hack your $p$-value into being as small as possible.

$\endgroup$
  • 7
    $\begingroup$ "some of your audience will consider this a trick to hack your $p$-value" — Indeed, I've reviewed 4 papers recently, and of the 2 that used a 1-tailed test, they only used a single 1-tailed test each (the rest were all 2-tailed), with no a priori justification, and the $p$ was such that the test succeeded, but would've failed had the test been 2-tailed. $\endgroup$ – Kodiologist Mar 13 '17 at 15:55
  • 1
    $\begingroup$ I agree that one-tailed tests tend to look (and are usually perceived as) suspicious (@Kodiologist), but in principle I think in most circumstances they would make more sense then two-tailed tests, not less. Most of the reasonable scientific hypotheses are directional. $\endgroup$ – amoeba Mar 13 '17 at 20:35
  • 1
    $\begingroup$ I like the argument that people use $5\%$ because it is the most commonly used significance level, and it is the most common because Fisher used it, and Fisher used it because he found from experience that about $2$ standard deviations was useful. So if you do not use a two-sided test, then you need a proper justification for a $5\%$ significance level beyond common usage $\endgroup$ – Henry Mar 13 '17 at 21:04
  • $\begingroup$ @Henry Thanks for pointing that out. If there was a .05 convention for two-tailed tests and a .025 convention for one-tailed tests, problems like the one described bei Kodiologist would disappear (ok, I see why we don't do that). $\endgroup$ – Bernhard Mar 14 '17 at 10:36
  • 1
    $\begingroup$ That is another way to look at it and right now I couldn't say, if one is more convincing than the other. I guess, Henry has the strongest point in saying, that it breaks down to conventions. Chosing a good $\alpha$ level each time wisely depending on the importance of the scientific claim would be more important then the question of number of tailes. $\endgroup$ – Bernhard Mar 14 '17 at 11:30
3
$\begingroup$

You have to consider how the $p$-value was determined. That will dictate how you should move between the $p$-value you have and the one you want. In general, a $p$-value is the proportion of possible values (e.g., test statistics, mean differences, etc.) that are as far away or further than your value under a given distribution.

In standard textbook presentations, the first step is to compute the area under the curve to the left of the value as you move from $-\infty$ to $\infty$. It may help to examine the figure below. If that is all you did (e.g., by looking up the $p$-value in a table, or what your software did), that constitutes the lower one-tailed $p$-value. If you wanted the upper one-tailed $p$-value, you would subtract that value from $1$ (not double or halve it).

$\hspace{12mm}$ enter image description here

However, it is pretty standard that statistical software will default to providing a two-tailed $p$-value. (To move from the textbook value above, you would multiply the $p$-value by two, if $<.5$—such as working with the white area in the figure above, or subtract from $1$ and then multiply by two, if $>.5$—such as working with the gray area above.) If you are starting from a two-tailed $p$-value, and you wanted to compute a one-tailed $p$-value, you need to determine which tail you are in relative to the tail you want to assess. Start by dividing the two-tailed $p$-value by $2$. Then, if your result is in the tail you are after (e.g., you want to know if $\bar x$ is significantly less than $\mu_0$ and $\bar x < \mu_0$), you are done. If your observed value is in the wrong tail, subtract the quotient from $1$.

It may help to walk through a simple calculation. For simplicity, imagine you test an observed sample mean (say, $.75$) against a null population mean value (call it $0$), when the population is known to be normally distributed and the standard deviation is known to be $1$. Your software returns a two-tailed $p$-value (viz., $.453$) and you want...

  • the two-tailed $p$-value: Stop, you're already there.
  • the $p$-value for the test that $\bar x > \mu_0$: $.453/2 = .227$.
  • the $p$-value for the test that $\bar x < \mu_0$: $.453/2 = .227\quad \Rightarrow \quad 1-.227= .773$.

If your software returned one of the above one-tailed $p$-values, and you wanted to determine the two-tailed $p$-value, you would reverse one of the processes listed above depending on whether you were in the tail you wanted or not.

$\endgroup$
2
$\begingroup$

In ALL cases, $\alpha$ is some standard value like 0.05, and you do not alter it for the directionality of the test. Do not halve it, double it, etc. What is always true is that $p$ is some rejection area, and you are testing whether the area is smaller than some standard threshold (e.g. 0.05). What changes is how you calculate $p$.

I always think of $p$ as a function of the cumulative distribution function (CDF) of the test $T$ statistic you calculated - that is, a function of $F(T)$, which is the integral of the PDF of the test statistic's theoretical distribution from $-\infty$ up to your $T$. Here's the crib sheet for the $p$-values, after which I will explain them:

Left-tailed: $p = F(T)$

Right-tailed: $p = 1-F(T)$

Two-tailed: $p=\begin{cases} T<0: & 2\cdot F(T)\\ T>0: & 2\cdot(1-F(T)) \end{cases}$

In all cases, you are trying to calculate the area of rejection. For left-tailed, it's just the CDF mentioned above. For right-tailed, it's the opposite: everything BUT the left-area, hence the $1-F(T)$. For two-tailed, it's slightly tricky. You're trying to create a rejection area of two symmetrical pieces. For a $T$ that is negative and thus lies to the left of the standardized distribution's mean of 0, you have something like a left-tail leading up to it. So, you take the left-tail calculation and double it. For a $T$ that is positive and thus lies to the right of the standardized distribution's mean of 0, you have something like a right-tail going past it. So you take the right-tail calculation and double it.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.