# What is the minimum viable cell size for 2x2 ANOVA?

I have a 2x2, between-subjects experimental design (2 independent variables (IVs) with 2 levels each) and one dependent variable (DV). My data are unbalanced and an interaction between the IVs seems likely, so I plan to use ANOVA with Type III Sums of Squares to test whether my DV is different across one of my IVs while controlling for any influence of the other IV (or the interaction, if present).

My question is this: what is the minimum number of data points that I need in each of my 4 cells, to ensure that ANOVA likely won't give spurious results? I'm aware of Simmons et al.'s recommendation (2011) of having at least 20 data point in each cell, but that recommendation only serves to control the test's rate of false negatives. What I'm more worried about is that with a small enough cell size, the statistics on which ANOVA is based probably won't be very reliable, and so the results of the test would be unreliable as well (in terms of either false negatives or false positives). Are there any papers or texts out there that have studied and made recommendations regarding this concern?

-
How severely unbalanced are the data? –  DoctorAmbient Mar 14 at 18:23
–  gung Mar 14 at 21:07
@DoctorAmbient, I'll let you judge the severity. In one data set, the counts across 4 cells are 16, 17, 13, 30. In another data set, the counts across 4 cells are 13, 10, 8, 14. –  Tyne Mar 19 at 10:45

There isn't really any absolute minimum except in a trivial sense (if you won't try to test for the interaction, the minimum $n_{ij}$ will be $1$, if you do want to test for interactions, the minimum cell size might be $2$). Instead, there are two issues here:
1. The first is the question of the robustness of the ANOVA to the violation of assumptions. Like all linear models (regression, $t$-tests, etc.), the ANOVA assumes that the data within each cell (i.e., the residuals) are independent, have equal (homogenous) variance, and are normally distributed. In truth, some of these assumptions are less necessary than others. For instance, with enough data you don't really need the within-cell distributions to be normal. However, what constitutes 'enough data' depends on how far from normal those distributions are. Thus, the further you are from normality, the more data you need. But there is another twist here, namely that with fewer data it is harder (or even impossible) to assess whether the assumptions of the ANOVA are met. So with fewer data per cell, you really are going by blind faith. If the assumptions are not met, then you can have increased type I error rates.
2. The second issue is the question of power. The probability of getting significance is a function of the size of the effect and the amount of data you have. If the effects are large enough, you will have good power even if you have only 1 datum per cell. I suspect effects that large are uncommon, though. Thus, you need to determine how large of an effect you want to be able to detect with what power (etc.), and calculate your $N$ accordingly.
Thanks @gung. I avoided mentioning robustness in my post because it seems to be an extremely loose concept. Can you provide (or link to) a detailed description of the relationship between $n_{ij}$ and the deviation of cell $ij$'s distribution from normality? As a rough example, the information that I seek might be represented as a height-map over a 2D plane, where the two dimensions of the plane are $n_{ij}$ and the deviation of cell $ij$'s distribution from normality, and the height from the plane represents the likelihood of ANOVA's type 1 error rate being inflated. –  Tyne Mar 19 at 10:09