0
$\begingroup$

A p-value is "the probability for a given statistical model that, when the null hypothesis is true, the statistical summary would be the same as or of greater magnitude than the actual observed results." Source

Thus, the p-value is defined as $\mathbb P_r(X ≥ x|H )$ for a one sided (right tailed) test. For a two sided (double tailed) test, this probability is doubled, meaning $ p = 2 \mathbb P_r(X ≥ x|H ) $ and then compared to $\alpha$ (e.g .05). However, how can this two sided p-value be interpreted as the probability as defined above? The real probability of getting a stat. summary of greater magnitude than the observed result (e.g. +2) is $ \mathbb P_r(X≥2|H)$ and not $2 \mathbb Pr(X≥2|H)$. Yet, statisticians double the right-hand p-value. There is, of course, an error probability on the left side too ($\alpha/2$), but this should be of no concern for us, as our observed value is positive and thus creates a probability only on its right side to $ +\infty $ below the curve.

The question, therefore, is: Would it not be more mathematically accurate to compare $ \mathbb P_r(X≥2|H)$ to $\alpha/2$ $(=.025)$ instead of $2 \mathbb P_r(X≥2|H)$ to $\alpha$ $(=.05)$ in a two sided test? The results would of course be the same in symmetric distributions, but not in asymmetric ones!

Furthermore, if we say that p for double tailed tests is indeed $2 \mathbb P_r(X≥x|H)$, then this $p$ is not the probability of $X$ being of greater magnitude than $x$, but rather this probability times $2$.

$\endgroup$
  • 3
    $\begingroup$ One of my intentions in writing stats.stackexchange.com/a/130772/919 was to help people understand two-tailed tests (and even more complicated situations), so please take a look. Your quotation, although from a reliable source, is correct only in certain textbook situations. It is not correct as a characterization of p-values generally. $\endgroup$ – whuber Jun 29 '18 at 20:18
3
$\begingroup$

I would prefer to consider extreme rather than magnitude

If you are expecting the value $0$ for something thought to have a symmetric distribution but in fact see the value $2$

then other values that you might have seen which would be as extreme as $2$ or more extreme than $2$ would include all values greater than or equal to $2$ and all values less than or equal to $-2$

so the probability of seeing $2$ or a more extreme value can be expressed as $\mathbb P(|X| \ge 2) = \mathbb P(X \ge 2) + \mathbb P(X \le -2) = 2\mathbb P(X \ge 2)$ by symmetry

In asymmetric distributions, you need to decide what extreme means in the context of your experiment

If you had a lightbulb which had an exponentially distributed lifetime L expected to be $1000$ hours, you might decide that extreme means that seeing a lifetime of $4000$ hours or more would be as extreme as seeing a lifetime of about $18.5$ hours or less, or you might take a different view. You could still say $\mathbb P(L \ge 4000) + \mathbb P(L \le 18.5) \approx 2\mathbb P(L \ge 4000) \approx 0.0366$

You should decide what you mean by as extreme or more extreme before you see the actual observation, and this is especially important with a distribution believed to be asymmetric

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.