I have 200-some variables with 25,000 observations in each of 4 categories. For each variable individually, I need to identify where we have variability between categories. Therefore, I need an ANOVA procedure with relevant post-hoc testing.

I looked at normality and variance - I have all non-normally distributed residuals and some with homogenous variance and some without homogenous variance (I used D'Agostino's normality test and Q-Q plots and Levene's test).

For those variables that are non-normal and homogenous, I've used Kruskal-Wallis to identify the variables that have variability between categories and Dunn's test to identify which categories differ.

I'm not sure how to proceed with non-normal data with heterogenous variance. I've read a lot of papers and posts and don't feel like I'm any closer to getting an answer. Should I just run the same tests (KW & Dunn's) on both the heterogenous and homogenous variables and call it good? Would I be better off violating the normality assumption and running Welch's ANOVA & T-Tests on variables with heterogenous variance? Or should I stick with non-parametric methods and, if so, how will the Dunn's test be impacted by the heterogenous variance? Or is there a better post-hoc? (Some materials seem to suggest Games-Howell in this case? But it doesn't have an implementation in Python, so if there's an alternative...)

I've also read that Levene's test isn't accurate with very large sample sizes and, in that case, should I just treat everything like it meets the homogenous variance assumption?

Thanks in advance.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.