I am currently running some mixed ANOVAs for a 2x2 design (two groups, with pre- and post- test). Some of my variable are fine, but some of them violate assumptions for parametric testing, e.g. non-normal, skewed, kurtosis. Furthermore, my sample is not really large enough to be assuming I can ignore violations due to the central-limit theorem: I have 24 in each group, so 48 at pre- and 48 at post-.
As a result, I am running my mixed using the WRS2 package in r, specifically with the
bwtrim() function, which works on trimmed means (I am using a trimming of 10% each side) and with the
sppbi() functions, which work by boot-strapping the results. However, when I do this, I do not get equivalent results with the two methods. For example, when I run bwtrim on my variable ABmelt, I get:
so we can see that there is a significant pre-post main effect change, but no main effect of group, and no interaction term. However, when I run the
sppbb() function, I do NOT get a significant main effect of time:
Call: sppbb(formula = OUTCOME_VARIABLE ~ GROUPING_VARIABLE * PrePost, id = ID, data = my_data)
Test statistics: Estimate PRE-POST -0.2057
Test whether the corresponding population parameters are the same: p-value: 0.134
In general, my results using the trimmed means are much more similar to my results running "normal" (non-robust) mixed ANOVA. So my question is this: which robust method should I be using? Is it a trade-off with robust-ness and power, or do we base or decision based on properties of the distributions?
In the published WRS2 papers (e.g. see here: https://dornsife.usc.edu/assets/sites/239/docs/WRS2.pdf) they seem to be using the bootstrapping primarily to look at more complex mixed-ANOVAs where there are more groups (3x4 design), but that doesn't really seem to be a reason to use one over the other.
For harmony across my analyses, I was wanting to use a robust method across all of my variables (I have around 10)- from those violating all assumptions (highly skewed, ceiling effect) to those that aren't really that problematic. Of course, if this is making me lose too much power, then I run into the risk of type II (instead of type I) error.
Any thoughts and suggestions appreciated! Thank you :)