3
$\begingroup$

My question is how to calculate type II error $\beta$?

  • Suppose I want to test $H_0: \mu=0$ vs $H_1: \mu=1$ (I need to calculate type II error $\beta$, so I need to fix a $\mu$, say 1, in $H_1$).

  • Suppose the distribution for $H_0$ is $F_0$, $H_1$ is $F_1$, where $E[\xi]=0$ if $ \xi\sim F_0$, $E[\xi]=1$ if $\xi\sim F_1$.

  • Now I create an estimator for $\mu$, say $\bar{X}_n$, and a test statistics $S_n=\frac{\bar{X}_n-E[F_0]}{\sigma}=\frac{\bar{X}_n-0}{\sigma}=\frac{\bar{X}_n}{\sigma}$ (let's assume $\sigma$ is known).

  • Now I create a reject($H_0$) rule: $S_n>b$.

  • Type II error is calculated as $P_{F_1}(S_n>b)$

My questions are(want to verify three things):

  • The above construction logic is correct, right?

  • The distribution in "$P_{F_1}(S_n>b)$" is $F_1$, right?

  • [most care about] The $S_n$ in "$P_{F_1}(S_n>b)$" should use $F_0$ to calculate, right?

    • I mean, no matter type I or type II error I am calculate, I always need to use $F_0$ to calculate the test statistics, right?

    • I mean, $S_n$ is always $\frac{\bar{X}_n-E[F_0]}{\sigma}$ in type I or type II error calculation, but not $\frac{\bar{X}_n-E[F_1]}{\sigma}$ in calculating $\beta$, right?

    • Or, this should not be a problem, because test statistics is just a function of sample and should not involve parameters?

$\endgroup$
2
  • 1
    $\begingroup$ Type II error is not to reject the null hypothesis when it is false, i.e $H_1$ is true. I think you should use $F_1$ to calculate P but not $F_0$ as you have written $P_{F_1}(S_n>b)$. You also can refer to power calculation whic is based on $H_1$ parameter, and Type II $\beta$ = 1-power $\endgroup$
    – Deep North
    Aug 27 '15 at 4:32
  • $\begingroup$ Thank you! You are right. I made a mistake. It is $P_{F_1}(S_n\le b)$ for the type II error. $\endgroup$ Aug 28 '15 at 13:53
2
$\begingroup$

Denote $\mathcal{F}^{(0)}(\mu=\mu_0,\sigma=\sigma_0)$ be the distribution under the null hypothesis and $\mathcal{F}^{(1)}(\mu=\mu_1,\sigma=\sigma_1)$ under $H_1$, so you have a test statistic $X$ and you want to test

$H_0: X \sim \mathcal{F}^{(0)}(\mu=0,\sigma=\sigma_0)$ versus $H_1: X \sim \mathcal{F}^{(1)}(\mu=1,\sigma=\sigma_1)$

The way you describe it, you want to perform a one-sided test, and you define the critical region in the right tail. So after you have chosen a confidence level $\alpha$, you will use the distribution $\mathcal{F}^{(0)}(\mu=0,\sigma=\sigma_0)$ to find the quantile value $q_{\alpha}^{(0)}$ such that $P^{(0)}(X \ge q_{\alpha}^{(0)})=\alpha$ ( I am assuming continuous distributions). The superindex $(0)$ indicates that the probabilities are measured under $\mathcal{F}^{(0)}$, so you need the null distribution $\mathcal{F}^{(0)}$ to define the critical region, i.e. the quantile $q_{\alpha}^{(0)}$.

From a sample you can observe an outcome $x$ for the random variable $X$ and the null will be rejected when $x \ge q_{\alpha}^{(0)}$. In other words your test will decide that $H_1 \textrm{ decided as true} \iff x \in [q_{\alpha}^{(0)};+\infty[$.

The power of your test is the probability that $H_1$ is decided as true whenever $H_1$ is true, so the power is the probability that $X \ge q_{\alpha}^{(0)}$ whenever $H_1$ is true, this is the probability that $X \ge q_{\alpha}^{(0)}$ when the true distribution is $\mathcal{F}^{(1)}$ or the power $\mathcal{P}$ is

$\mathcal{P}=P^{(1)}(X \ge q_{\alpha}^{(0)})$

Where the superindex $(1)$ indicates that the probabilities are computed under $\mathcal{F}^{(1)}$ So the power is measured with $\mathcal{F}^{(1)}$ but you need the value of $q_{\alpha}^{(0)}$ which is computed with $\mathcal{F}^{(0)}$.

I used the power $\mathcal{P}$ and the type II error $\beta$ is $\beta=1-\mathcal{P}$.

In your case

You are right when you say that ''The distribution in "$P_{F_1}(S_n>b)$" is $F_1$''

However, in order to find $b$ you will have to use the $F_0$. In fact, the $b$ is the analog of $q_{\alpha}^{(0)}$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.