4
$\begingroup$

Consider a confusion table summarizing the outcomes of $n$ tests of hypothesis $H_0$ for $n$ independent experiments:

             accept H0    reject H0
H0 is true      a             b
H0 is false     c             d

Here $a$, $b$, $c$ and $d$ are number of times we accepted/rejected $H_0$ given that $H_0$ was true/false, and $a+b+c+d=n$.

If we divide numbers in the table by $n$, we will get estimates of $P(TN)$, $P(FN)$, $P(FP)$, $P(TP)$:

             accept H0    reject H0
H0 is true     P(TN)         P(FP)
H0 is false    P(FN)         P(TP)

Here, $P$ is probability, $TN$ - true negative decision, $FN$ - false negative decision, $FP$ - false positive decision, $TP$ - true positive decision.

Now, the power of test is defined as $P(TP)$; $P(FN)$ is designated by $\beta$. An article on statistical power in Wikipedia says that power = $1 - \beta$, so that $P(TP) + P(FN) = 1$, so that $d/n + c/n \approx 1$. I don't see how it is possible in this setup, so what did I get wrong? Similarly, $P(TN)$ + $P(FP)$ should not be 1, or should it?

$\endgroup$
2
  • 2
    $\begingroup$ Power is actually defined as $1-\beta$, so it's not remotely surprising that power+$\beta$=1 (as per your title) - it's a tautology. Your confusion arises elsewhere. $\endgroup$
    – Glen_b
    Dec 19, 2014 at 1:54
  • $\begingroup$ We never accept the H0. We fail to reject. $\endgroup$
    – Aman J
    Jul 17, 2021 at 19:07

1 Answer 1

3
$\begingroup$

The confusion here is that you're mixing up conditional and unconditional probability.

Power is a conditional probability: P(reject H0|H0 false).

Similarly $\beta$ is P(fail to reject H0|H0 false).

These conditional probabilities add to 1. That is: $\frac{c}{c+d} + \frac{d}{c+d} =1$.

Similarly, for the top row: $\frac{a}{a+b} + \frac{b}{a+b} =1$ (where the probabilities represent $1-\alpha$ and $\alpha$).

This is NOT the same as saying $\frac{c}{a+b+c+d} + \frac{d}{a+b+c+d} =1$; that's plainly not true.

power of test is defined as $P(TP)$

Beware! For it to be a true positive (rather than just "a positive"), you've specified that the null is false.

The sensitivity, or true positive rate (TPR) = TP/P = TP/(TP+FN). That's what power is.

That is, if by P(TP) you mean P(reject|H0 false), then I agree that's the power, but then it isn't $\frac{d}{a+b+c+d}$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.