# The meaning of two sided (double tailed) p-values

A p-value is "the probability for a given statistical model that, when the null hypothesis is true, the statistical summary would be the same as or of greater magnitude than the actual observed results." Source

Thus, the p-value is defined as $$\mathbb P_r(X ≥ x|H )$$ for a one sided (right tailed) test. For a two sided (double tailed) test, this probability is doubled, meaning $$p = 2 \mathbb P_r(X ≥ x|H )$$ and then compared to $$\alpha$$ (e.g .05). However, how can this two sided p-value be interpreted as the probability as defined above? The real probability of getting a stat. summary of greater magnitude than the observed result (e.g. +2) is $$\mathbb P_r(X≥2|H)$$ and not $$2 \mathbb Pr(X≥2|H)$$. Yet, statisticians double the right-hand p-value. There is, of course, an error probability on the left side too ($$\alpha/2$$), but this should be of no concern for us, as our observed value is positive and thus creates a probability only on its right side to $$+\infty$$ below the curve.

The question, therefore, is: Would it not be more mathematically accurate to compare $$\mathbb P_r(X≥2|H)$$ to $$\alpha/2$$ $$(=.025)$$ instead of $$2 \mathbb P_r(X≥2|H)$$ to $$\alpha$$ $$(=.05)$$ in a two sided test? The results would of course be the same in symmetric distributions, but not in asymmetric ones!

Furthermore, if we say that p for double tailed tests is indeed $$2 \mathbb P_r(X≥x|H)$$, then this $$p$$ is not the probability of $$X$$ being of greater magnitude than $$x$$, but rather this probability times $$2$$.

• One of my intentions in writing stats.stackexchange.com/a/130772/919 was to help people understand two-tailed tests (and even more complicated situations), so please take a look. Your quotation, although from a reliable source, is correct only in certain textbook situations. It is not correct as a characterization of p-values generally.
– whuber
Jun 29 '18 at 20:18

I would prefer to consider extreme rather than magnitude

If you are expecting the value $$0$$ for something thought to have a symmetric distribution but in fact see the value $$2$$

then other values that you might have seen which would be as extreme as $$2$$ or more extreme than $$2$$ would include all values greater than or equal to $$2$$ and all values less than or equal to $$-2$$

so the probability of seeing $$2$$ or a more extreme value can be expressed as $$\mathbb P(|X| \ge 2) = \mathbb P(X \ge 2) + \mathbb P(X \le -2) = 2\mathbb P(X \ge 2)$$ by symmetry

In asymmetric distributions, you need to decide what extreme means in the context of your experiment

If you had a lightbulb which had an exponentially distributed lifetime L expected to be $$1000$$ hours, you might decide that extreme means that seeing a lifetime of $$4000$$ hours or more would be as extreme as seeing a lifetime of about $$18.5$$ hours or less, or you might take a different view. You could still say $$\mathbb P(L \ge 4000) + \mathbb P(L \le 18.5) \approx 2\mathbb P(L \ge 4000) \approx 0.0366$$

You should decide what you mean by as extreme or more extreme before you see the actual observation, and this is especially important with a distribution believed to be asymmetric