Doubling &/or halving p-values for one- vs. two-tailed tests

Question

Let's say I'm doing a two-tailed hypothesis test at 5% significance level and get a test statistic that corresponds to a p-value of $0.03$. As it is two-tailed I double it and therefore, as $0.06 > 0.05$, I fail to reject the null hypothesis.

However, let's say now I'm checking for a result strictly greater than the mean so it's now a one-tailed test. At 5% significance level, do I now reject my null hypothesis as $0.03 < 0.05$ or as we're only doing a one-tailed test do I halve $0.05$ and check against that?

Answer 1

If you do a two-tailed test and computation gives you $p=0.03$, then $p<0.05$. The result is significant. If you do a one-tailed test, you will get a different result, depending on which tail you investigate. It will be either a lot larger or only half as big.

$\alpha=0.05$ is the usual convention, no matter whether you test one- ode two-tailed. You don't halve that (except maybe in Bonferroni-correction, which is not the topic here). Thus yes, sometimes a one-tailed test will give you a significant result where the two-tailed does not. However, this is not how things work: You have to always determine upfront, whether you consider a one- or a two-tailed test appropriate as you have to determine your $\alpha$-level upfront. Then you calculate the $p$-value for that test and there are no more degrees of freedom how to test or what to compare the $p$-value to. If you determine on the sidedness of your test depending on whether you like the result, this is not good scientific practice.

That being said, there is hardly ever a situation where it is appropriate to test one-tailed. In far most circumstances it would be worth communicating a significant result in both directions. If you test one-tailed, some of your audience will consider this a trick to hack your $p$-value into being as small as possible.

Answer 2

You have to consider how the $p$-value was determined. That will dictate how you should move between the $p$-value you have and the one you want. In general, a $p$-value is the proportion of possible values (e.g., test statistics, mean differences, etc.) that are as far away or further than your value under a given distribution.

In standard textbook presentations, the first step is to compute the area under the curve to the left of the value as you move from $-\infty$ to $\infty$. It may help to examine the figure below. If that is all you did (e.g., by looking up the $p$-value in a table, or what your software did), that constitutes the lower one-tailed $p$-value. If you wanted the upper one-tailed $p$-value, you would subtract that value from $1$ (not double or halve it).

$\hspace{12mm}$

However, it is pretty standard that statistical software will default to providing a two-tailed $p$-value. (To move from the textbook value above, you would multiply the $p$-value by two, if $<.5$—such as working with the white area in the figure above, or subtract from $1$ and then multiply by two, if $>.5$—such as working with the gray area above.) If you are starting from a two-tailed $p$-value, and you wanted to compute a one-tailed $p$-value, you need to determine which tail you are in relative to the tail you want to assess. Start by dividing the two-tailed $p$-value by $2$. Then, if your result is in the tail you are after (e.g., you want to know if $\bar x$ is significantly less than $\mu_0$ and $\bar x < \mu_0$), you are done. If your observed value is in the wrong tail, subtract the quotient from $1$.

It may help to walk through a simple calculation. For simplicity, imagine you test an observed sample mean (say, $.75$) against a null population mean value (call it $0$), when the population is known to be normally distributed and the standard deviation is known to be $1$. Your software returns a two-tailed $p$-value (viz., $.453$) and you want...

• the two-tailed $p$-value: Stop, you're already there.
• the $p$-value for the test that $\bar x > \mu_0$: $.453/2 = .227$.
• the $p$-value for the test that $\bar x < \mu_0$: $.453/2 = .227\quad \Rightarrow \quad 1-.227= .773$.

If your software returned one of the above one-tailed $p$-values, and you wanted to determine the two-tailed $p$-value, you would reverse one of the processes listed above depending on whether you were in the tail you wanted or not.

Answer 3

In ALL cases, $\alpha$ is some standard value like 0.05, and you do not alter it for the directionality of the test. Do not halve it, double it, etc. What is always true is that $p$ is some rejection area, and you are testing whether the area is smaller than some standard threshold (e.g. 0.05). What changes is how you calculate $p$.

I always think of $p$ as a function of the cumulative distribution function (CDF) of the test $T$ statistic you calculated - that is, a function of $F(T)$, which is the integral of the PDF of the test statistic's theoretical distribution from $-\infty$ up to your $T$. Here's the crib sheet for the $p$-values, after which I will explain them:

Left-tailed: $p = F(T)$

Right-tailed: $p = 1-F(T)$

Two-tailed: $p=\begin{cases} T<0: & 2\cdot F(T)\\ T>0: & 2\cdot(1-F(T)) \end{cases}$

In all cases, you are trying to calculate the area of rejection. For left-tailed, it's just the CDF mentioned above. For right-tailed, it's the opposite: everything BUT the left-area, hence the $1-F(T)$. For two-tailed, it's slightly tricky. You're trying to create a rejection area of two symmetrical pieces. For a $T$ that is negative and thus lies to the left of the standardized distribution's mean of 0, you have something like a left-tail leading up to it. So, you take the left-tail calculation and double it. For a $T$ that is positive and thus lies to the right of the standardized distribution's mean of 0, you have something like a right-tail going past it. So you take the right-tail calculation and double it.