5
$\begingroup$

I am studying blocking in ANOVA and I am wondering about the following scenario.

Suppose we did a Generalised randomised block design. Suppose SSBL = 0 and it also had not interaction effect with the treatment.

Now if I do the F test on the hypothesis $H_0: \text{treatment effects are zero}$ then I will see that my p-values here will be larger because the denominator degrees of freedom are lesser whereas the F test statistic value $F^* = \dfrac {MSTR}{MSE} $ will remain the same. Does this mean that my type I error rate has increased due to choosing the wrong blocking variable? I read somewhere that the power of F test will decrease in such a scenario (apologies I am unable to find that link now). But if Type I error rate increase then type II error decreases which means increase in power of test.

So I am unable to understand how the error rates are affected by use/misuse of blocking. Can someone please make it a bit clear for me?

$\endgroup$

2 Answers 2

2
$\begingroup$

You will not get an increase in type I errors, even if $SSBL = 0$. It is true that your denominator will have fewer degrees of freedom, but the $SSE$ will be identical because $SSBL$ is $0$. Thus, you will divide $SSE$ by a smaller number, yielding a larger $MSE$. This in turn means that the denominator of your $F$ ratio is larger, yielding a smaller $F$ statistic. Since your $p$-value gets smaller as your $F$ value gets larger, your $p$-value will get larger when your $F$ statistic is smaller. There are a lot of steps to follow here, but the end result is that your $p$-value will be larger. Here's a schematic:
$$ \big(SSE\ {\rm same}\ \&\ df_E \downarrow \big) \quad \Rightarrow \quad MSE \uparrow \quad \Rightarrow \quad F \downarrow \quad \Rightarrow \quad p \uparrow $$
Suppose that the null is true, and that $SSBL = 0$. Because the null is true, you will make a type I error if $p<\alpha$. Your alpha is fixed a-priori, so if your $p$-value is larger, it is less likely to be $<\alpha$. Therefore, the probability of a type I error is lower. That is, you will have lower power, and possibly a lower type I error rate, but certainly not a higher one.

$\endgroup$
2
  • $\begingroup$ Suppose NULL is true, then how come this larger p-value doesnt signify larger type-I error rate? $\endgroup$
    – Durin
    Jan 2, 2015 at 20:56
  • 1
    $\begingroup$ @Durin, suppose that the null is true. Then you will make a type I error if $p<\alpha$. Your alpha is fixed a-priori, if your $p$-value is larger, it is less likely to be $<\alpha$. Therefore, the probability of a type I error is lower. $\endgroup$ Jan 2, 2015 at 20:59
2
$\begingroup$

Are you sure that you type I error increases? You talk about the specific case where the SSBL=0, but in reality if the true blocking has no effect, the observed SSBL is very unlikely to be exactly 0, so your F statistic will be different. The change in the F statistic and change in the degrees of freedom should cancel each other out to give the same type I error rate (on average).

Restricting the blocking sum of squares to be exactly 0 would be like doing a 1 sample t-test and restricting the sample mean to be exactly the same as the hypothesized mean, unlikely to ever be observed and yes that artificial case does effect the type I error. But reality will have a difference in the observed mean/sum of squares.

You can test this out by simulation. Generate some data where the blocking has no effect (don't force the sum of squares to 0 though) and analyze it both with and without the blocking factor. Repeat the process a bunch of times to see how often you reject the null. If the null is true then this gives a comparison of type I error rates (if all the other assumptions hold then both should be close to alpha), if the null is false then this will let you compare power and you should see more power by not blocking (you may be surprised by the size of the difference in power).

$\endgroup$
2
  • $\begingroup$ Thank you for replying. So what is the theoretical explanation behind increase in power? $\endgroup$
    – Durin
    Jan 2, 2015 at 19:14
  • 2
    $\begingroup$ @Durin, when there is no blocking effect then blocking will cost you degrees of freedom without any gain in the MSE, so will have lower power. When there is a blocking effect then using blocking will reduce your MSE increasing the F statistic and thereby giving more power. $\endgroup$
    – Greg Snow
    Jan 2, 2015 at 20:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.