Colleagues and I conducted an experiment with participants randomly allocated to each condition using survey software. Unfortunately, the cell sizes ended up being more unbalanced than we anticipated: Out of around 1743 participants, each randomized to one of 10 conditions (this was a 2x5 factorial design), 3 conditions ended up with just under 160 participants each, while two conditions wound up with 200 participants each. A quick simulation suggests only about a 6% chance of winding up with 3 or more conditions with fewer than 160 participants with this kind of randomisation regime, so we seem just to have gotten a bit unlucky.
I have seen advice to use type II or type III sums-of-squares for unbalanced ANOVA; we were planning to use type III anyhow as we are estimating an interaction (and checking that results are similar with type II if there is no interaction). Is this sufficient? Are there any rules-of-thumb we might follow, or analyses/simulations we might consider conducting, to assess whether this level of imbalance is likely to pose a problem?
Edit: Some additional context that may be helpful: For the analysis in question, we are testing the effect of four different kinds of information (and a no-information control), accompanying an either positive or negative COVID PCR test result, on participants' agreement with the proposition that the test recipient should self-isolate (on a 0 to 100 scale). We hypothesized an interaction with differences between positive/negative test results for some kinds of information but not others. And we got one, p < .001 - but wanting to ensure that this isn't somehow an artifact of our unbalanced numbers.
