6
$\begingroup$

Oftentimes I see people encouraged to do orthogonal designs "so they can have a unique partition of the sums of squares" but I don't have any intuition on why that should matter or how it relates to interpretability. I'm thinking here particularly, but not exclusively, about the context of a One-Way ANOVA. Orthogonal designs produce balanced covariate distributions. It can be impractical, costly, and time consuming to set up an orthogonal.

Is there necessarily a loss of efficiency or interpretability if the experiment is unbalanced? If so, to what extent is this observed?

Some aspects of this question were explored, albeit in a rather shorthand way, in the answers to this question.

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ You are asking a very good question. Most people are told about so-called optimal design and ascribe dogmatically to its principles. This paper from Stephen Senn is a great read, illuminating, and argues your case here. In summary, balance is a little oversold: it can prevent some catastrophic disasters, but merely adjusting for the factors in a multivariate model has comparable efficiency in most cases. $\endgroup$
    – AdamO
    May 31, 2018 at 15:44
  • 2
    $\begingroup$ You might get some insight into the issue of efficiency & imbalance from my answer here: How should one interpret the comparison of means from different sample sizes? $\endgroup$
    – gung - Reinstate Monica
    May 31, 2018 at 16:58

1 Answer 1

Reset to default
2
$\begingroup$

In a multi-way ANOVA (e.g., a two-way ANOVA), unbalanced designs have the disadvantage that the main effects are not independent (orthogonal) to the interactions of which they are apart. As such, you get different estimates for the test of the main effects depending on whether you fit a Type I, II, or III sums-of-squares model. In a balanced design, this is not an issue. This is the justification for balanced factorial designs when doing experimental work where you have control over such things. This is not an issue in a one-way ANOVA because there is only one factor (i.e., no interaction). However, balanced sample sizes across groups in a one-way ANOVA will maximize statistical power, assuming a fixed total N and all other things being equal. This balance also helps the model be robust to violations of the equal variances assumption. That said, increasing the sample size in just one group does increase statistical power. This increase will be less than you would achieve by dividing this increase equally across groups. For example, a design with sample sizes of 30-30-30 (N=90) across three groups will have more power than a design with 20-30-40 (N=90). However, a design with 30-30-60 (N=120) will have more power than a design with 30-30-30 (N=90), but less than a design with 40-40-40 (N=120).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.