I'm using SPSS. I have a multi-level categorical IV and a few continuous DV's.
My main analysis goals are to test the effect of the IV on the DV, and then to follow-up with pairwise comparisons of level 1 of the IV to each of the other levels.
However, Levene's test for homogeneity of variance is significant for all of them, and the sample sizes are very unequal across levels of the IV.
To deal with the heterogeneity of variance and unequal sample sizes, I conducted Welch tests in place of standard one-way ANOVA's, and followed up with independent sample t-tests, consulting the adjustment for unequal variances where appropriate (equivalent to a Welch t-test).
However, I have now realised that my main IV is confounded with some of the measured demographic factors (age, sex, region) that also have an effect on the DV, so I'd like to control for that. To do this, it would make sense to run an ANCOVA, with pairwise comparisons using estimated marginal means, rather than raw means. However, ANCOVA (of course, as a variant of ANOVA) assumes equal sample sizes and homogeneity of variance.
Is there a way I can simultaneously (i) correct for these assumption violations, while (ii) controlling for the possible effects of covariates? If not, which is the better route to take?
I read that heterogeneity of variance is especially a problem if the variance ratio (largest variance/smallest variance) is greater than 2 and can perhaps be ignored if it's lower than that (though I don't know whether this is still true if you have unequal sample sizes, like I do). The variance ratios for my 3 DV's are 2.01, 1.54, and 2.34.
(When I said Levene's tests were significant, that's just the 'based on mean' version - the 'based on median' version is not significant. But I don't really know what that means or if that has any real relevance here.)
I should say that running it in both ways doesn't appear to change the fact that the IV has a significant effect (p<.05) on the DV or which of the pairwise comparisons turn out to be significant, at least for the primary DV's - but I want to get it right, and carry forward the correct procedure for some of the secondary DV's (where it could in principle make a difference).