# F-test: How to use type I error to compute type II error?

Here is the conclusion from the reference

https://www.real-statistics.com/chi-square-and-f-distributions/power-of-two-sample-variance-testing/

Let $$s_1^2$$ and $$s_2^2$$ represent the variances of two independent samples of size $$n_1$$ and $$n_2.$$ We know that $$\dfrac{s^2_1/\sigma^2_1}{s^2_2/\sigma^2_2}\sim F_{n_1-1,n_2-1}.$$ Let $$x_{crit}$$ be the critical value based on the null hypothesis with significance level $$\alpha:$$ $$x_{crit} = F_{n_1-1,n_2-1}^{-1}(\alpha)$$ For the one-tailed test

$$H_0: \sigma_1^2 ≥ \sigma_2^2\ i.e.\ \lambda = \dfrac{\sigma_1^2}{\sigma_1^2}\geq1,$$ we have the beta value: $$\beta = F_{n_1-1,n_2-1}(x_{crit}/\lambda).$$

I don't know how to obtain the value of $$\beta$$ when we know $$\alpha?$$ If we use $$T/F$$ to represent accept/reject $$H_0;$$ $$1/0$$ to represent $$H_0$$ is true/false. Then we have conditions:

$$P(F|1) = \alpha = F_{n_1-1,n_2-1}(x_{crit});$$ $$P(1) = F_{n_1-1,n_2-1}(\lambda).$$ We want to compute: $$\beta = P(T|0)?$$

I don't think the conditions are enough to compute $$\beta.$$

• You can compute $\beta$ only if you have a specific numerical value of $\lambda$ in mind. Power can be computed only as probability of detecting a particular alternative. Mar 2, 2021 at 14:40
• when we have value $\lambda = \lambda_0,$ how can we represent $\beta?$ Mar 2, 2021 at 15:56
• Perhaps see this. Mar 2, 2021 at 17:33

Power of 2-Sample F test for Population Variances.

Null and one-sided alternative hypotheses: Suppose you have random samples, both of size $$n_i=n_2=20$$ from normal populations with variances $$\sigma_1^2, \sigma_2^2,$$ respectively. You want to use their sample variances to test whether to reject $$H_0: \sigma_1^2 = \sigma_2^2$$ against $$H_a: \sigma_1^2 > \sigma_2^2.$$ Sometimes the hypotheses are written as $$H_0: \Lambda =\frac{\sigma_1^2}{\sigma_2^2} = 1$$ against $$H_a: \Lambda > 1.$$

Null distribution: If $$H_0$$ is true, then the distribution of the ratio $$R$$ of sample variances is given by $$R = \frac{S_1^2}{S_2^2} \sim F(n_1-1, n_2-1),$$ which is $$R \sim F(19,19)$$ in our example. It seems natural to reject $$H_0$$ in favor of $$H_a$$ if $$R$$ is sufficiently large. [The variance ratio is sometimes called F in computer printouts, but I am using $$R$$ here to avoid confusion between the test statistic and its distribution.]

Critical value: In particular, one can use printed tables of $$F$$ distributions (or software) to find the critical value $$c$$ for a one-sided test at the 5% level. In our example $$c = 2.168$$ is found using R as below. Roughly speaking $$R = S_1^2/S_2^2$$ needs to exceed about $$R = 2.17$$ in order for $$H_0$$ to be rejected.

c = qf(.95, 19, 19);  c
[1] 2.168252


One may say that the significance level $$\alpha = 0.05 - 5\%$$ is the probability of a Type I error: $$\alpha = P(\mathrm{Rej}\,|\,\Lambda=1).$$

Type II error and power: The question now arises, how likely are we to reject $$H_0,$$ if $$\Lambda = \frac{\sigma_1^2}{\sigma_2^2} = 4.$$ That is, how likely are we to reject if the first population variance is four times as big as the second population variance?

The power of the test to detect this ratio $$\Lambda = 4$$ in population variances is probability of rejecting $$H_0$$ for the particular alternative with $$\Lambda = 4;$$ we might designate the power as $$P(\mathrm{Rej}\,|\,\Lambda=4) = \pi(4) = \pi(\Lambda=4),$$ depending upon what information is obvious from the immediate context.

A Type II error occurs if $$H_a$$ is true and we fail to reject. We might designate the probability of a Type II in this situation as $$P(\mathrm{Fail\,to\,Rej}\,|\,\Lambda=4) = \beta(4) = \beta(\Lambda=4) = 1 - \pi(\Lambda=4).$$

In general, for independent normal samples, $$\frac{S_1^2/\sigma_1^2}{S_2^2/\sigma_2^2} = \frac{S_1^2}{S_2^2}/\Lambda = R/\Lambda \sim F(n_1-1,n_2-1).$$

Power computation for example: In our example, it follows that $$\beta(4)=0.0956, \pi(4)=0.9044$$ can be found in R as shown below. (Many printed tables of F distributions lack the detail to find these exact probabilities.)

1 - pf(c/4, 19, 19)
[1] 0.904437
pf(c/4, 19, 19)
[1] 0.09556305


So with two samples of size twenty, we are reasonably sure (power about 90%) that an F-test will detect that the first population has four times the variance as the second.

Software procedures: Many kinds of statistical software have 'power and sample size' procedures used to find the power for given sample sizes or the sample sizes needed to achieve a desired power. In Minitab software, one specifies the ratio $$\lambda = \sqrt{\Lambda}$$ of population standard deviations one hopes to detect. Many of these procedures give results only for the 'balanced' case where $$n_1 = n_2.$$ Minitab output for our example follows:

Power and Sample Size

Test for Two Standard Deviations

Testing (StDev 1 / StDev 2) = 1 (versus >)
Calculating power for (StDev 1 / StDev 2) = ratio
α = 0.05
Method:  F Test

Sample
Ratio    Size     Power
2      20  0.904437

The sample size is for each group.


Simulation: It is sometimes convenient to approximate the power provided by particular sample sizes by using a simulation. [Notes: (a) Sample sizes are equal here, but they need not be. Population means are not relevant to the situation and are both set to $$0.$$ (c) With 100,000 iterations, it is reasonable to expect about two place accuracy for power.]

set.seed(302)
r = replicate( 10^5, var(rnorm(20,0,2))/var(rnorm(20,0,1)) )
c = qf(.95, 19, 19)
mean(r > c)
[1] 0.90388  # aprx power


Note: If you have the budget for $$10$$ additional subjects, it is better to increase both samples to size $$25,$$ increasing the power of $$95.3\%,$$ rather than to increase one sample to $$30$$ and leave the other at $$20,$$ obtaining power $$94.5\%,$$ Balanced designs are more efficient.

set.seed(303)
r = replicate( 10^5, var(rnorm(25,0,2))/var(rnorm(25,0,1)) )
c = qf(.95, 24, 24)
mean(r > c)
[1] 0.95346

set.seed(304)
r = replicate( 10^5, var(rnorm(30,0,2))/var(rnorm(20,0,1)) )
c = qf(.95, 29, 19)
mean(r > c)
[1] 0.94472